This article is part of our AI Agents for Business hub. Want help applying it? Talk to eSEOspace.
AI agent evals are repeatable tests that check whether an agent did a job correctly. You give the agent a task, let it work, then score the result with defined rules.
They exist because agents do not behave the same way twice. A test that passes once proves very little. This guide is part of our AI agents for business hub, and it was checked against Anthropic's and vendors' documentation on 9 November 2026.
Key Takeaways
- An eval gives an AI an input and applies grading logic to measure whether it succeeded.
- Agent output varies between runs, so each task is run several times.
- Anthropic says 20 to 50 simple tasks drawn from real failures is a great start.
- Capability evals show what an agent can do, and regression evals show whether it broke something.
- Automated scores need human checks, because a grader can be wrong too.
What an eval is
Anthropic, which sells AI models, published a detailed guide to agent evals on 9 January 2026. It says: "An evaluation ("eval") is a test for an AI system: give an AI an input, then apply grading logic to its output to measure success."
Think of it as a practical exam for a new hire. You set the same work each time, you know what a good result looks like, and you mark it the same way.
The words you will hear
These terms come from Anthropic's guide. The plain-English examples are ours.
| Term | Anthropic's meaning | Example |
|---|---|---|
| Task | A single test with defined inputs and success criteria | "Refund this order under our 30-day policy" |
| Trial | One attempt at a task | Run number 3 of 10 |
| Grader | Logic that scores some aspect of the agent's performance | Did the refund amount match? |
| Transcript | The complete record of a trial | Every message and tool call |
| Outcome | The final state at the end of the trial | The refund exists in the system |
The difference between transcript and outcome matters. An agent can say "I refunded it" while nothing changed in your system. Check the outcome, not just the claim.
Why each task runs more than once
Traditional software gives the same answer to the same input. An agent may not. Anthropic's guide says: "Because model outputs vary between runs, we run multiple trials to produce more consistent results."
This changes what a score means. Anthropic describes two ways to read repeated runs: success in at least one attempt, or success in every attempt. For anything touching customers or money, the second reading is the one to care about.
Three kinds of grader
Anthropic groups graders into three types and recommends combining them. Its guide says: "We recommend choosing deterministic graders where possible, LLM graders where necessary or for additional flexibility, and using human graders judiciously for additional validation."
- Code-based graders check facts automatically, such as whether a record was updated. They are fast and cheap.
- Model-based graders use another AI to judge open-ended work, such as tone. They handle nuance but cost more and can vary.
- Human graders are people reviewing a sample. They are slower and more expensive, but they catch what the others miss.
Capability evals and regression evals
Anthropic separates evals by purpose. Capability evals ask what the agent can do well. Its guide says: "They should start at a low pass rate, targeting tasks the agent struggles with and giving teams a hill to climb."
Regression evals ask whether the agent still handles past tasks. Anthropic says they should have a nearly 100% pass rate.
You need the second kind the moment an agent is live. A model update, a changed prompt or a new tool can break something that used to work.
How to start
Anthropic's guide says: "In reality, 20-50 simple tasks drawn from real failures is a great start." It adds that more mature agents may need larger, harder evals.
- Collect real cases, including jobs people got wrong and jobs the agent got wrong.
- Write down what a correct result looks like for each one.
- Include cases where the agent should refuse or do nothing.
- Score the outcome in your system wherever you can.
- Run each task several times and record the results.
- Read a sample of the transcripts yourself.
Step 3 is easy to skip. Anthropic warns that "One-sided evals create one-sided optimization." For example, an agent only tested on when to search may end up searching for almost everything.
Step 6 matters too. Anthropic says: "You won't know if your graders are working well unless you read the transcripts and grades from many trials."
When evals are worth the effort
Anthropic's guide describes teams without evals as stuck in reactive loops. It says: "Absent evals, debugging is reactive: wait for complaints, reproduce manually, fix the bug, and hope nothing else regressed."
That matters for owners, because a problem found by a customer costs more than one found in a test. Survey data suggests the practice is still uneven. In LangChain's survey of 1,340 respondents, 52.4% ran offline evals and 37.3% ran online evals.
LangChain sells agent tools and its respondents are self-selected builders. Treat that as a picture of agent builders, not of small businesses in general.
Where to run evals
You may not need a separate product. These were checked against vendor documentation on 9 November 2026.
- Copilot Studio lists evaluations that "validate quality with test sets and a shared grader library before and after you publish."
- n8n's pricing page lists light evaluations on all cloud plans, and metric-based evaluations for 1 workflow on Starter and unlimited on Pro.
- OpenAI is closing its tool. Its deprecations page says existing evals became read-only on 31 October 2026. It adds: "The Evals dashboard and API are scheduled to shut down." The date listed is 30 November 2026.
That last point is a reason to keep your test cases in a format you own. A list of tasks and expected outcomes in a spreadsheet survives a vendor change. We cover the wider shutdown in our Agent Builder article.
Evals do not replace watching the live agent
A test set only covers what you thought to test. Anthropic's guide says: "The most effective teams combine these methods: automated evals for fast iteration, production monitoring for ground truth, and periodic human review for calibration."
So pair evals with agent observability. For a broader checklist on judging an agent, see how to evaluate an AI agent.
Next step
Pull 20 real cases from your inbox or help desk this week, and write the correct result next to each. That list is the start of your eval set, whichever tool you use. If you want it built into an agent project, see our AI agent development service.
Frequently asked questions
What is the difference between an eval and a demo?
A demo shows one run that went well. An eval runs a defined set of tasks several times and scores them the same way. Only the second tells you how often the agent works.
How many test cases do I need?
Anthropic says 20 to 50 simple tasks drawn from real failures is a great start. It adds that more mature agents may need larger and harder sets to detect smaller effects.
Can AI grade AI?
Yes, and it is one of the three grader types Anthropic describes. Anthropic notes that model-based graders are non-deterministic and need calibration with human graders, so check a sample by hand.
How often should I run evals?
Run them whenever something changes, such as the model, the prompt, the tools or the connected data. Regression evals exist to catch backsliding after those changes.
Do I need a developer to run evals?
Not for a first set. You can write tasks and expected results in a spreadsheet and score a small batch by hand. Automated scoring needs either someone technical or a platform with built-in evaluations.
