Here's how most AI systems get approved for launch: someone senior types in a few questions, the answers look impressive, and the room nods. That's not evaluation — that's a magic show. The demo reveals the best case; production is decided by the average case and the worst case, and nobody in that room saw either. The teams whose AI keeps working after launch all share one unglamorous habit: they measure. Not with a research lab, not with a platform purchase — often starting with twenty rows in a spreadsheet. Here's how it works.
#Vibes vs. evidence
Vibe-checking
- Tries a handful of favorite prompts
- Judged by whoever's in the room that day
- The best answer dominates the impression
- Regressions slip in silently with every change
- Confidence built on anecdotes
Systematic evals
- Runs hundreds of real cases, every time
- Scored against criteria written down in advance
- Measures the average and the worst case
- Every change gets a pass/fail verdict before shipping
- Confidence built on numbers you can show the board
#First, define what 'good' means
You can't measure quality you haven't defined. For any AI use case, 'good' decomposes into a handful of concrete dimensions — and writing them down is half the value of the whole exercise:
- Correctness — is the answer factually right, and grounded in the sources it was given?
- Completeness — did it address the whole request, or quietly skip the hard part?
- Format — does the output match the structure downstream systems and people expect?
- Tone and safety — is it on-brand, appropriate, and free of things you'd never want screenshot?
- Cost and latency — is it fast enough and cheap enough at the volume you'll actually run?
#The eval loop
Evaluation isn't a phase before launch — it's a loop that runs for the life of the system. Five steps, repeating:
- 1Define
Write down what a right answer looks like for your use case.
- 2Collect
Build a test set from real inputs — including the ugly, weird ones.
- 3Score
Automated checks, an AI judge for nuance, humans for spot checks.
- 4Compare
Every prompt or model change runs the full suite before shipping.
- 5Monitor
Production failures feed straight back into the test set.
#Scaling the scoring
Once the suite outgrows manual review, layer the scoring. Cheap automated checks catch the objective failures: is the format valid, are required fields present, does the cited source exist? A second AI — an 'LLM judge' — grades the subjective dimensions like relevance and tone at scale. And humans keep auditing a sample, because the judge itself needs calibrating: if its grades diverge from your experts', fix the judge before trusting the dashboard.
“A demo tells you what the AI can do. An eval tells you what it will do. You ship on the second one — or you find out in production, in front of customers.”
If you have an AI system today and no test set, that's the highest-leverage afternoon available to you: pull twenty real inputs, write down what good looks like, and score. Whatever the result, you'll know something you didn't this morning — and every improvement from here on can be proven instead of hoped.
