This article is part of our AI Agents for Business hub. Want help applying it? Talk to eSEOspace.
AI agents make mistakes often enough that you should plan for them, but we found no verified real-world error rate for agents in business. Lab tests range from about 66% task success on one test to 2.5% on another.
Those numbers look contradictory because they measure different things. This guide is part of our AI agents for business hub. It explains what each test shows and what it doesn't, and it was checked on 9 November 2026.
Key Takeaways
- We found no verified field error rate for AI agents in business.
- Stanford's 2026 AI Index reports about 66% task success on the OSWorld test. It says agents still fail roughly 1 in 3 attempts on structured benchmarks.
- A Salesforce research benchmark found about 58% single-turn success and about 35% in multi-turn settings.
- Other tests of longer, real-world style work found much lower results.
- The only number that matters for your decision is the one you get on your own tasks.
The numbers side by side
These results come from different tests, at different dates, on different models. Do not average them, and do not compare them directly.
| Test | What it measures | Reported result | Date |
|---|---|---|---|
| OSWorld, via Stanford's AI Index | Agents doing real computer tasks across operating systems | From 12% to about 66% task success | 2026 report |
| CRMArena-Pro | Business and CRM scenarios | About 58% single-turn, about 35% multi-turn | May 2025 |
| TheAgentCompany | Tasks in a simulated software company | 30% completed autonomously by the top agent | Version 3, September 2025 |
| Remote Labor Index | Real freelance-style projects | 2.5% automation rate for the top agent | October 2025 |
The older results used older models. Scores on tests like these change quickly, so treat each one as a snapshot from its date.
What each test actually says
OSWorld. Stanford's report says: "AI agents made a leap from 12% to ~66% task success on OSWorld, which tests agents on real computer tasks across operating systems, though they still fail roughly 1 in 3 attempts on structured benchmarks."
CRMArena-Pro. Its paper says: "Experiments reveal leading LLM agents achieve only around 58% single-turn success on CRMArena-Pro, with performance dropping significantly to approximately 35% in multi-turn settings." This work comes from Salesforce's research group, and Salesforce sells agents.
The same paper reports over 83% single-turn success on workflow execution, so some kinds of business work scored much better. It also says: "Furthermore, agents exhibit near-zero inherent confidentiality awareness; though targeted prompting can improve this, it often compromises task performance." That matters if your agent can see private customer data.
TheAgentCompany. The authors write: "We test baseline agents powered by both closed API-based and open-weights language models (LMs), and find that the most competitive agent can complete 30% of tasks autonomously." They add that a good portion of simpler tasks could be solved autonomously. Harder, long-horizon tasks were out of reach of the systems they tested.
Remote Labor Index. Its abstract says: "AI agents perform near the floor on RLI, with the highest-performing agent achieving an automation rate of 2.5%." These are large projects judged on the whole deliverable, which is a much harder bar.
Why the numbers disagree
Four things change the result.
- Length of the job. Short tasks score better than long ones, because each extra step is another chance to go wrong.
- Conversation. CRMArena-Pro's score fell from about 58% to about 35% when the task went multi-turn.
- How success is defined. Finishing a sub-step is different from delivering finished work a client would accept.
- The date. A model from 2025 is not the model you would buy today.
So "How accurate are agents?" has no single answer. A better question is how accurate an agent is on a specific job, with specific tools.
Why we found no real-world rate
A lab controls the tasks and the scoring. In a business, tasks vary, not every output gets scored and mistakes can go unnoticed.
Vendors may publish accuracy numbers for their own products. Those are worth reading, but they are the vendor's numbers on the vendor's chosen tasks.
Stanford's report does record that "Documented AI incidents rose to 362, up from 233 in 2024." That counts AI incidents in general and is not an error rate.
Our guide to how benchmarks work explains how to read scores like these. For current comparisons between models, see our reliability leaderboard.
What kinds of mistakes to expect
Plain averages hide what a mistake actually costs you. Watch for these.
| Type of mistake | Example | Why it matters |
|---|---|---|
| Wrong answer | States the wrong refund policy | Reaches a customer as fact |
| Wrong action | Updates the wrong record | Creates cleanup work |
| False success | Says it finished when it did not | Hides the problem |
| Compounding error | Builds later steps on a bad result | Damage grows |
| Over-acting | Does more than it was asked | May cross a boundary |
The examples are ours. Anthropic, which sells AI models, describes the compounding risk in its engineering guide. It says: "The autonomous nature of agents means higher costs, and the potential for compounding errors."
How to find your own error rate
You can get a real number for your job. It takes a small amount of work.
- Collect 20 to 50 real cases where you know the right result.
- Run each case several times, because results can vary between runs.
- Score whether the outcome was right, not whether the agent said so.
- Read a sample of the runs yourself.
- Repeat after any change to the model, prompt or tools.
Anthropic's guide to evals says: "Because model outputs vary between runs, we run multiple trials to produce more consistent results." Our article on AI agent evals explains the method.
How many mistakes are acceptable?
That depends on what a mistake costs and how easily you can catch it. A draft that a person reviews can tolerate plenty of errors. A payment that goes out automatically can tolerate very few.
Compare the agent with the realistic alternative. If your current process also has errors, measure those too, so you are comparing real numbers. For ways to catch problems, see human-in-the-loop agents.
Next step
Before you buy, ask the vendor to run your real cases and show you every failure. If you want an agent built and tested on your own work, see our AI agent development service.
Frequently asked questions
Are AI agents accurate enough for business?
For some jobs, yes, especially when a person reviews the output. For unattended work with real consequences, test first. Accuracy depends on the task, the tools and how you define success.
What is the error rate of AI agents?
No verified real-world error rate exists that we could find. Lab tests range widely because they measure different tasks. Test your own job to get a number that means something.
Do AI agents make more mistakes than people?
We found no reliable direct comparison for business work. Measure both on the same tasks if the decision matters to you.
Why do AI agents say they finished when they did not?
Language models generate plausible text, so a confident statement is not proof. That is why you should check the actual result in your system, such as whether the record really changed.
Are AI agents getting better?
On some tests, yes. Stanford's report shows a jump from 12% to about 66% on OSWorld. It also says agents still fail roughly 1 in 3 attempts on structured benchmarks.
