How much can you trust a model to grade its own work

Once you're generating output at any volume, checking it by hand stops scaling and the obvious move is to have a model grade the output instead. An LLM-as-judge. The question I kept hitting is the one that decides whether the whole thing is trustworthy or just automated false confidence: how much human validation does that judge actually need before you can believe its verdicts?

The tempting answer is zero. Write a good rubric, point a capable model at the outputs, read the scores. I went looking for whether that holds, and it's refuted twice over, from two independent directions. First, judges pass synthetic perturbation tests, the manufactured obvious errors, and then collapse on the realistic domain-specific errors that are the entire reason you're evaluating. They look calibrated on easy checks and fail on hard ones. Second, and this is the sharper finding, judges agree with human experts mainly on the items the judge could have answered correctly itself. In a study with professional graders in a finance domain, judge-expert agreement was high only on questions the judge already knew, and expert-written reference answers were what fixed the rest. The paper is titled "No Free Labels," which is the thesis in three words. In any domain where correctness is the hard part, there is no free lunch on labelling.

There's a nastier detail underneath. When a judge is wrong, its errors are not random noise, they correlate with the generator. Models causally favour text they recognise as their own, and the deeper driver is familiarity: a judge over-scores text that reads in a style it finds familiar, regardless of who wrote it. LLM errors also correlate across different models, so stacking judges into an ensemble doesn't save you; they agree while being wrong on the same blind spot. Correlated error is the worst failure shape you can have in a checker, because it produces confident false ship-signals, not the visible randomness you'd catch.

So zero-label is out. But the reason people avoid building evals at all is a fear that the alternative is weeks of labelling, and that fear is wrong in the other direction. The human gate is small, front-loaded, and compressible. The working minimum is 50 to 100 labels before you even write the judge's criteria, and 20 is explicitly too few to understand your own data. At a couple of minutes per short output, 100 labels is two to five hours of one qualified person's time. The whole calibration set costs low hundreds of dollars. One team hit over 90 percent judge-expert agreement in three calibration passes. This is days of part-time effort, not weeks.

The real constraint is that you can't fully write your evaluation criteria in advance, because you discover what "good" means while you're grading real outputs. Graders provably rewrite their own rubrics mid-way through the first pass. That's exactly why no zero-label workflow can work: there's nothing to validate the judge against until a human has decided what good looks like on actual output, and that decision only forms by looking at actual output.

The shape I've settled on, drawn from the practitioners who've done this most carefully, has a few firm rules. Do error analysis on real traces before you pick any metric or tool; the failure taxonomy you observe is the eval spec, and every automated check should trace back to a failure mode you actually saw, not one you imagined. Label the first 50 to 100 cold, before showing the human any model output, because pre-labelling contaminates their judgement with the model's answer. Judge one failure mode per judge, binary pass or fail with a written critique before the verdict, not a one-to-five Likert score and not one compound judge trying to weigh everything at once. Graduate autonomy per dimension, not wholesale: a generic criterion earns trust in days once it agrees with your labels tightly enough, while a domain-specific one that needs real expertise may stay human-in-the-loop far longer. And accept that faithfulness checking catches maybe 60 to 80 percent of defects, not everything, so treat the scores as signal rather than truth.

The framing that dissolved the bottleneck for me was to stop thinking humans-then-model, as if you label for weeks and then switch the humans off. Run the model judge from week one, and point your five hours of human attention at the slice of cases the judge is least confident about. The humans don't get removed. They get aimed at the items that buy the most calibration per hour. That's a system you can actually trust, and it costs a long afternoon to stand up.