Classical model validation leans on things an LLM does not give you. You cannot inspect a closed weight matrix, you usually cannot retrain a vendor's foundation model, and you often cannot reproduce a result bit for bit because the provider ships a new checkpoint without telling you. The validation question changes from "is the math correct" to "does this system behave correctly, on our data, under our conditions, and can we prove it still does."
Evals answer that question. A well-built eval suite is not a developer's test harness bolted on at the end. It is the primary body of validation evidence for a model you cannot open. SR 26-2 (issued April 17, 2026) supersedes SR 11-7 for in-scope models, but it expressly excludes generative and agentic AI, so LLM validation falls under general risk-management expectations rather than the model-risk framework. That gap is precisely why eval-based evidence carries so much weight here. This lesson covers the three pieces that carry the weight: golden sets, benchmarks, and challenger models.
The golden set is your ground truth
A golden set is a curated collection of inputs paired with known-good outputs, labeled by people who understand the domain. It is the thing you measure against. Everything else in your eval program depends on the golden set being honest.
Three rules separate a credible golden set from a checkbox.
First, build it from real production failures, not synthetic happy-path examples. The cases that matter are the ones where the model already got something wrong or nearly did. A golden set assembled only from clean, obvious inputs will report high scores and validate nothing.
Second, size it for the task, not for comfort. Teams running unit-style evals in development typically curate 200 to 500 examples, which is enough to catch regressions on a focused use case without becoming impossible to maintain. A retrieval-heavy or high-materiality system needs more, and the higher its risk tier (covered in module 3), the more coverage your validator should expect.
Third, version it and record provenance. Who labeled each case, when, against what instructions, and why it carries a particular risk tag. This is the traceability that frameworks like the NIST AI Risk Management Framework and ISO 42001 ask for, and it is what lets a validator trust the score instead of the team that produced it.
Worked example: a KYC document-extraction assistant
Suppose we deploy an LLM that reads scanned onboarding documents and extracts beneficial-owner names, dates, and ownership percentages. We assemble a golden set of 350 cases: 120 drawn from production tickets where extraction was wrong or ambiguous, 150 representative clean cases, and 80 deliberate edge cases (rotated scans, two people sharing a surname, percentages that should sum to 100 but do not).
Each case carries the document, the correct structured output, the labeler's identity, and a risk tag. We then measure exact-match on names and dates, and tolerance-banded accuracy on percentages. The golden set is not the model. It is the yardstick we hold the model against, and we can hand it to an independent reviewer without explaining ourselves.
Benchmarks tell you what the golden set cannot
Your golden set proves performance on your task. Benchmarks place the model in a wider context: how it compares to alternatives, where it sits against public reference tasks, and whether known weaknesses apply to your use.
Use benchmarks for two jobs. One is selection, comparing candidate models before you commit. The other is bounding, understanding documented failure modes so your golden set deliberately probes them.
Be skeptical of leaderboard scores. Frontier models now saturate many public benchmarks, which means a near-perfect headline number tells you almost nothing about your specific task. A model that tops a reasoning leaderboard can still misread a rotated KYC scan. Treat external benchmarks as a coarse filter and a source of failure hypotheses, never as a substitute for measuring on your own data.
When the judge is itself a model
Many teams score open-ended outputs with an LLM-as-judge because human grading does not scale to continuous testing. This is reasonable, but the judge is a model too, and it carries its own risk that a validator will ask about.
The biases are documented and large. Studies have measured position bias as high as roughly 75 percent preference for the first response in a pairwise comparison, and on harder bias benchmarks such as JudgeBiasBench, frontier models have shown error rates above 50 percent. Length bias and authority bias push judges toward verbose, citation-heavy answers regardless of correctness.
You manage this the way you manage any model in the chain. Calibrate the judge against human labels on a slice of the golden set, swap positions in pairwise prompts to cancel position bias, break subjective calls into binary sub-decisions, and record the agreement rate. A judge you have measured is evidence. A judge you have assumed is a liability.
The challenger model replaces retraining
Under SR 11-7, a core validation technique was the benchmark or challenger model, an independently built alternative used to test whether the primary model's outputs are reasonable. With an LLM you cannot open or retrain, the challenger becomes more useful, not less, because it is one of the few ways to apply effective challenge from the outside.
A challenger does not need to be another large model. It can be a different foundation model, a smaller fine-tuned model, a retrieval-plus-rules pipeline, or even a deterministic heuristic. What matters is that it is built independently and that disagreement between it and the primary model flags cases for review.
In the KYC example, our primary model is a large general-purpose LLM. Our challenger is a cheaper, narrower extraction model run on the same documents. We route every input through both. Where they agree, confidence is high. Where they disagree on a name or an ownership percentage, the case escalates to a human and lands in next cycle's golden set. The challenger gives us a continuous, automated form of effective challenge without ever touching the primary model's weights.
SR 26-2 emphasizes that effective challenge depends on competence and sufficient independence to maintain objectivity, easing some prescriptive independence mechanisms while still requiring genuine objectivity. A challenger model is exactly that kind of evidence: an independent voice that disagrees on the record, captured in numbers a validator can audit.
Closing takeaway
For a model you cannot open or retrain, evals are the validation. Build a golden set from real failures and version it like a model artifact. Use benchmarks to select and to bound, never as a stand-in for your own data. Run a challenger to apply effective challenge continuously, and measure your judge before you trust it. Together these produce the one thing validation has always required, evidence that survives independent review, even when the model underneath stays sealed.