This article is part of our AI Agent Guides. See how models compare this week on the AI Agent Leaderboard, a ranking of rankings refreshed every week.
A model is the AI brain, and a harness is the software that gives it tools and runs its loop. An agent is the two working together. So when someone says "this AI scored well", the honest follow-up question is: in which harness, at which settings?
This matters when you buy. The same model in two different products can behave differently and cost very different amounts.
Disclosure: eSEOspace is a member of the Claude Partner Network.
Key Takeaways
- Model: the trained AI that reasons and writes. Harness: everything around it. Agent: model plus harness.
- Research from September 2026 found the harness can change cost a lot, while success rates on the tested benchmarks moved much less.
- Settings count too. Reasoning effort, time limits and hardware can all change a result.
- Benchmarks use different setups, so read which one produced a result before comparing.
- See current results on the AI Agent Leaderboard. Selecting a model shows the harness used.
Three terms in plain English
| Term | What it is | Office comparison | Examples, as of November 2026 |
|---|---|---|---|
| Model | The trained AI that reasons and generates text | A new hire's brain and education | Claude, GPT and Gemini model families |
| Harness | Code that supplies tools, manages context and runs the loop | The desk, software, checklist and manager | Claude Code, Codex CLI, Pi |
| Agent | A model running inside a harness | The employee actually doing the job | Any of the above models in any of those harnesses |
LangChain's guide to harnesses offers the shortest rule we have seen: if you are not the model, you are the harness. That covers code, configuration and execution logic.
The Claude Code documentation puts it in similar terms. Models reason, tools act, and the layer that provides the tools and manages what the model sees is the agentic harness. For the wider basics, start with what is an AI agent.
What a harness actually controls
A harness makes many decisions before the model thinks about your task. Each one can change results or the bill.
- Which tools exist. A harness might offer a handful, or dozens.
- What the model sees. It decides which files, notes and instructions load at the start.
- How long the instructions are. Longer starting instructions mean more tokens billed on each call. A token is a small chunk of text.
- How errors are handled. It decides what happens when a command fails.
- When to stop. It sets limits on steps, time or spending.
Tools also connect differently between harnesses, often through the Model Context Protocol. Our MCP guide and our list of AI coding agents with MCP go deeper.
How much does the harness really matter?
The best evidence we found is Arena's HarnessTax study, published in September 2026. It tested 21 model and harness pairs, using seven models and three harnesses: Claude Code, Codex CLI and Pi. It ran them on two open benchmarks, SWE-bench Lite and Terminal-Bench 2.0.
Arena's finding was less dramatic than the usual story. On those benchmarks, harness choice had little effect on task success, but it could significantly affect cost. The same model could reach similar success rates at up to five times the cost.
Some specifics, using a fixed price list dated 1 September 2026:
- On SWE-bench Lite, across shared models, Claude Code cost about 2.0 times as much as Pi and 1.6 times as much as Codex.
- On Terminal-Bench 2.0, Claude Code cost about 1.5 times as much as Pi.
- The average harness effect on success stayed within about 2% on SWE-bench Lite and about 5% on Terminal-Bench 2.0.
- Claude Code's average initial context was over 10 times Pi's, due to longer instructions and larger tool schemas.
Keep the limits in view. This was one study, on the coding benchmarks named above, with a sample of 30 tasks in its measurements. It says nothing about features such as permissions, review flows or team controls, which can justify paying more.
Why results can still differ
The harness is not the only thing that changes between setups. When two results for the same model disagree, check these first.
| What differs | Why it matters | Where to look |
|---|---|---|
| Harness | Changes cost and sometimes success | The "agent" column on the board |
| Reasoning effort | More thinking usually costs more and may help | Labels such as High or Max |
| Benchmark version | Later versions may be harder | The version number |
| Time limits | More time makes tasks easier | The methodology page |
| Hardware | Tight limits can fail agents for non-AI reasons | Often undisclosed |
Hardware deserves a special mention. Anthropic reported that infrastructure configuration alone produced a 6 percentage point gap on Terminal-Bench 2.0 between its most- and least-resourced setups. It suggests skepticism about leaderboard differences below 3 points until the setup is documented and matched.
Terminal-Bench's own team treats time limits the same way. It compares a changed timeout to giving one student longer on an exam. Its 4.0 release sets a flat 8-hour agent timeout for that reason and calibrates resources.
How benchmarks handle the harness
Benchmark publishers deal with this problem in different ways, and those choices shape what a result means.
- Fixed harness. Vals AI calls models through a harness it controls, so differences in scores reflect the model and not the scaffold.
- Native agents too. Vals says many of its benchmarks also run models inside coding agents, sometimes the model's own native agent.
- Model and agent listed together. Terminal-Bench's board has both a model and an agent column, so one model can appear more than once.
- Model isolated by experiment. Arena's method randomizes components to estimate each one's effect. It currently ranks only the orchestrator model, meaning the main model choosing the tools.
That is why our coding board lets you open a row to see the harness behind each result. Our methodology page explains how we keep each model's best configuration.
What this means when you buy
Do not compare "Model A vs Model B". Compare complete setups on your own tasks, with the same limits. A hypothetical example:
| Setup | Finished your 10 test tasks | Cost | Takeaway |
|---|---|---|---|
| Model A in Harness X | 8 of 10 | $40 | Baseline |
| Model A in Harness Y | 8 of 10 | $16 | Same outcome, much cheaper |
| Model B in Harness X | 9 of 10 | $70 | Extra task may or may not be worth it |
These numbers are invented, but the pattern matches what the Arena study describes. Here is a short checklist for your own comparison:
- Run each setup on the same real tasks, several times, since agent results vary.
- Record the harness, model, effort setting and time limit next to each result.
- Track cost per finished task, not price per token.
- Weigh what the harness adds: permissions, audit logs, review steps, integrations.
- Check whether your data rules allow each setup.
If you plan to build one yourself, frameworks supply much of the harness for you. See Claude Agent SDK for business for one example.
Frequently asked questions
Is Claude Code a model or an agent?
Claude Code is a harness, and with a model running inside it, an agent. Its documentation says it provides the tools and manages the context the model sees. The Claude models do the reasoning.
Can I use one vendor's model in another vendor's harness?
Often, yes. The HarnessTax study ran models across harnesses, and it reports that models may perform better with other harnesses than their own. Check the vendor's terms and supported options for your specific plan.
Does a better harness make a weak model strong?
The evidence above suggests not by much. In that study, harness choice moved success far less than it moved cost. A good harness helps you get value from a model, but it does not replace the model's ability.
Which should I choose first, model or harness?
Start with your constraints: budget, data rules and the tools you must connect. Those usually narrow the harness options. Then shortlist models from the board and test the combinations on real work.
