A golden dataset is not a giant pile of prompts. It is a small set of examples the team trusts enough to use when comparing changes.
Start with the product contract. List the main user jobs, dangerous failures, and ambiguous cases. Add successful production traces, corrected failures, and deliberately designed edge cases. For every example, store the input, necessary context, expected facts or behavior, category, and evaluation method.
Use deterministic assertions where possible. Use a clear rubric when several outputs could be valid. Have qualified humans resolve disagreements before calling an example “golden.”
Version the dataset. Prompts, policies, and products change; expected behavior may change too. Record why a case was added and which production incident it protects against.
Twenty carefully reviewed examples can be more useful than thousands of synthetic ones. The first goal is not statistical completeness. It is a stable baseline that catches failures the team genuinely cares about and grows whenever production teaches you something new.