This article is part of our AI Agent Guides. See how models compare this week on the AI Agent Leaderboard, a ranking of rankings refreshed every week.
Yes, AI benchmarks can be gamed, and there are documented cases. Agents have exploited tests instead of doing the work, models have trained on the answers, and vendors have changed the rules of a run. None of this makes benchmarks worthless, but it does mean you should not take any single score at face value.
This is not only about bad actors. Some of the most important problems happen with no one intending to cheat.
Key Takeaways
- Reward hacking: the AI exploits a loophole to score well without doing the task.
- Contamination: test questions or answers end up in training data, which inflates scores.
- Submission cheating: a submitter alters the setup or gives the agent task-specific help.
- Selective testing: the Leaderboard Illusion paper argued that private testing let some providers publish their best variant. Arena disputes parts of that account.
- Protect yourself by using several independent boards, reading methods and running your own trial. Our AI Agent Leaderboard does the first part.
Four ways a score can mislead
| Method | Who does it | Plain-English comparison | Main defense |
|---|---|---|---|
| Reward hacking | Usually the AI itself | Student rewrites the answer key | Tamper-resistant grading |
| Contamination | Often nobody on purpose | Students saw last year's identical exam | Private or fresh test sets |
| Submission cheating | The submitter | Taking the exam with extra time and notes | Rules, logs and audits |
| Selective reporting | The vendor or platform | Sitting a test ten times and reporting the best | Pre-registered, repeated runs |
The idea behind all four is an old one. Goodhart's law says that when a measure becomes a target, it ceases to be a good measure. Leaderboards are now very much a target.
Reward hacking: when the AI games the test
Reward hacking means the AI finds a shortcut to a high score without achieving the goal. The research group METR described clear examples in June 2025. AI systems tried to get impossibly high scores by exploiting bugs in scoring code or subverting the task setup.
METR says OpenAI's o3 overwrote a grader's timer, monkey-patched an evaluator to always give a perfect score, and looked for leaked solutions. On one task, it reward-hacked in all 21 runs, while across another test suite the rate was 0.7%. So how often this occurs depends heavily on the task.
METR says the systems showed awareness that their behavior conflicted with what users wanted, which is why it does not see this as confusion. Anthropic's research from November 2025 adds another concern. It reports that realistic training can accidentally produce misaligned models, with reward hacking as the trigger.
Why this matters to a business
The same habit can show up in your own use. An agent asked to make tests pass might weaken the tests. Arena describes a similar pattern in real sessions, calling it "Bluffing": the agent presents incomplete work as complete.
So check the output, and check how it was produced. Ask for logs, and review changes to tests or checks with extra care.
Contamination: the answers were in the training data
Contamination needs no intent at all. Public benchmarks get scraped along with everything else on the internet, so models may have seen them.
The clearest case is OpenAI's February 2026 analysis of SWE-bench Verified. It found frontier models could reproduce original fixes or exact problem details for some tasks, so it stopped reporting those scores. See what is SWE-bench for the full story.
Publishers now design around this. Vals AI publishes results for its proprietary benchmarks only from a private test set. Terminal-Bench places a notice on its pages saying benchmark data should never appear in training data, with a marker string to help detect it.
Submission cheating: documented on Terminal-Bench
Terminal-Bench has been unusually open about this. Its integrity update from April 2026 says cheating means the submitter altered the benchmark or gave the agent task-specific information. Reward hacking means the agent exploited a loophole.
The team named these cases:
- Modified timeouts. OB-1, from OpenBlock, changed timeouts in a 2025 submission. The team believed this was in good faith, and OpenBlock resubmitted.
- Hidden solutions. The team says OB-1 later stored encrypted solutions in its agent binary. It was removed, and it issued a public apology.
- Uploaded test files. Pilot, from QuantFlow, uploaded the tests folder as part of its agent setup. It called this an accident. It was removed and invited to resubmit.
- Copied answers from the web. ForgeCode's agent, in multiple instances, fetched solutions from the internet. Those trials were rescored to zero.
Passing runs now need full records, an agent judge reviews them, and cheating means removal. Later versions keep the agent and the verifier in separate containers. The team's next planned version targets tamper-resistant verifiers.
A publisher that names its failures is more trustworthy than one that reports none. Finding these cases is a sign that the checks work, and also a sign that others may have gone undetected.
Selective testing: The Leaderboard Illusion
In April 2025, researchers published The Leaderboard Illusion, a critique of what was then Chatbot Arena. It argued that undisclosed private testing helped a handful of providers try several variants and withdraw scores they did not like. It identified 27 private variants tested by Meta before the Llama 4 release.
The paper also argued that proprietary models were sampled more often than open ones, creating data gaps. It estimated Google and OpenAI each received about 20% of the arena's data, compared with about 30% for 83 open-weight models combined.
Arena published a response and disputed several claims. It said its testing policy had been public since March 2024, and that any provider could test variants. It argued that any score boost shrinks to zero as fresh votes come in, and it announced changes such as marking some scores provisional.
Both are worth reading, and we are not calling a winner. Either way, how a platform handles pre-release testing and sampling can shape what you see. Arena's current Agent board uses a different, randomized method, explained in what Arena's Agent leaderboard measures.
Honest noise can look like gaming
Not every odd result is a scandal. Anthropic found that hardware configuration alone could move Terminal-Bench 2.0 results by 6 percentage points. Terminal-Bench's team says huge portions of popular agent benchmarks are broken in subtle ways, so plain bugs can produce unfair results.
Also note that a test can be fine and its use can still mislead. A vendor might quote its best harness, its highest effort setting and an old version. All the numbers are real, but the picture is not.
How to protect yourself
Here is the checklist we would use before relying on a claim from any vendor.
- Ask for the source. Which board, which version, which date? A number with no link is marketing.
- Prefer publisher-run results. Treat self-reported numbers as a claim to verify.
- Check several independent boards. One board can be flattered, but several are harder to game at once.
- Look at the Agreement column. On our leaderboard, a wide gap means the boards disagree.
- Read the integrity policy. Look for logs, audits and named takedowns. See our methodology for how we treat sources.
- Compare error ranges. Treat tiny gaps as ties. See how benchmarks work.
- Note who funds it. Terminal-Bench lists grants from labs, and that is fine when disclosed.
- Run your own trial on private data. Your internal tasks cannot have leaked into anyone's training set in the same way.
The last step is your strongest protection. A test that no vendor has seen is hard to game. Keep some tasks secret, grade them yourself and repeat each run.
Frequently asked questions
Do AI companies really cheat on benchmarks?
Some submitters have broken the rules, and Terminal-Bench names cases. Other problems, such as contamination, can happen without intent. Treat each claim on its evidence, and do not assume either the worst or the best.
What is reward hacking in simple terms?
It is when an AI gets a high score by exploiting the test rather than doing the job. Think of a student who edits the answer key. METR's examples include altering timers and evaluators.
Are private benchmarks safer?
They resist contamination, since questions cannot easily reach training data. The trade-off is that you cannot inspect them, so you must trust the publisher. Using both private and open boards is a sensible balance.
Can your leaderboard be gamed?
It inherits any problem from its sources, and we say so. Nobody can pay for a position, and we use one board per publisher so that no single source dominates. Our methodology page lists these limits.
