LLM Evals: How to Test and Measure AI Applications

7 min readUpdated October 29, 2026

Evals are tests for AI behavior. Because LLM outputs vary, you can't check them with exact string matches alone. Instead you build datasets of realistic inputs and graders that score outputs, then run them on every change to prompts, models or retrieval.

Start from real failures

Read 50–100 real outputs by hand. Write down the failure types you see (wrong facts, bad format, missed citations, unsafe replies). Each failure type becomes a grader. This beats starting from generic benchmarks.

Three kinds of graders

Code-based: valid JSON, required fields, contains a citation, length limits, regex checks. Cheap and deterministic, so use them wherever possible.
LLM-as-judge: another model scores outputs against a written rubric. Great for tone and faithfulness, but watch for position bias, verbosity bias and self-preference.
Human review: the gold standard for high-stakes decisions and for creating ground-truth labels.

Metrics that matter

For classification, use precision, recall and F1, not just accuracy. For RAG, measure recall@k and MRR for retrieval, and faithfulness for generation. For agents, use task success rate, steps and cost per task.

Run evals in CI

Treat evals like unit tests: store the dataset in your repo, run it on every pull request that touches prompts or models, and block merges when the pass rate drops below your threshold. Keep a held-out set so you don't overfit your prompts to the test cases.

Practice what you just read

▶Practice it hands-onEvals: Testing LLM Systems ▶Practice it hands-onEvaluating RAG: Recall, MRR & Faithfulness ▶Practice it hands-onObservability: Traces, Latency & Cost

Frequently asked questions

How many eval examples do I need?

Start with 20–50 realistic cases covering your main failure types, then grow the set as you find new failures in production.

Can I trust LLM-as-judge?

Partly. Calibrate the judge against human labels, use specific rubrics, and swap answer order in pairwise comparisons to reduce bias.