This article is part of our AI Agent Guides. See how models compare this week on the AI Agent Leaderboard, a ranking of rankings refreshed every week.
Hire an AI agent developer who can show you how they test and contain an agent, not just how they build one. A good candidate asks about your process first and suggests something simpler when it fits. They will also prove the work on a small paid pilot.
An AI agent developer builds software that plans steps and uses tools to finish a job. It is a new specialty, so titles and portfolios tell you less than they would for web development. A demo proves little, because almost anyone can make an agent succeed once.
Disclosure: eSEOspace is a member of the Claude Partner Network. eSEOspace also offers MCP development, so weigh our advice accordingly.
Start by writing the brief
Do not start with resumes. You will get better candidates and better quotes if you write down the job first. One page is enough.
- The job: what the agent does, step by step, in today's words.
- The systems: every app and database it needs, and whether each has an API.
- The data: whether it touches health, legal, financial, or personal information.
- The risk: what the worst realistic mistake would cost.
- The measure: how you will know it is working.
Check first that you need to hire at all. Our build vs buy guide can help, since a ready-made agent may solve your problem for far less.
Where to look for candidates
Good candidates come from a few places, and each has a trade-off.
- Your network: A referral from someone who has run an agent in production is the strongest signal.
- Your software vendors: Many platforms keep partner directories. Partners know that platform well, and they may favor it.
- Freelance marketplaces: These offer a wide choice and fast starts. Expect to do more screening yourself.
- Agencies and consultancies: You get more roles under one contract. Ask who will do the work, because the sales lead may not.
- Open source contributors: Public code lets you see how someone works before you speak to them.
Use the same brief and questions for everyone, so you can compare answers directly.
Skills that actually matter
Look for these, in rough order of importance.
| Skill | What good looks like | How to check |
|---|---|---|
| Evaluation | Writes test cases and scores results over repeated runs | Ask to see a real test set |
| Security | Limits tool permissions and treats inputs as untrusted | Ask about prompt injection |
| System integration | Handles auth, retries, and rate limits | Ask about a past integration failure |
| Workflow design | Knows when not to use an agent | Ask when they talked a client out of one |
| Model judgment | Chooses models by cost per finished job | Ask how they would compare two models |
| Monitoring | Logs each step and tool call | Ask to see a trace |
| Communication | Explains limits in plain English | Notice how they answer you |
Security deserves extra attention. OWASP's Top 10 for LLM applications lists prompt injection and excessive agency among its risks. Prompt injection means text in an email or document tricks the agent into following hidden instructions. A serious developer can explain this without notes.
Framework names matter less than the thinking behind them. Anthropic's guide to effective agents says the most successful teams it worked with used simple, composable patterns rather than complex frameworks. Still, a developer should be able to explain why they would pick a tool such as the Claude Agent SDK or OpenAI Agents SDK. Neither is automatically right for your job, and each ties you to some vendor choices.
Questions to ask
About their experience
- Show me an agent you built that is running in production today. What does it do each week?
- What was its failure rate when it launched, and how did you measure it?
- Tell me about an agent project that failed or got canceled. What did you learn?
- When have you told a client they did not need an agent?
- Can I speak with a client you did not choose as your best example?
About how they would build yours
- What would you simplify in my brief?
- Which actions would you require human approval for, and why?
- What is the first small version you would ship?
- How would you test it before real customers or staff touch it?
- What happens when a tool call fails halfway through a job?
About security and data
- What credentials does the agent get, and how are they scoped?
- Where are prompts, outputs, and logs stored, and who can read them?
- How would you defend against prompt injection in untrusted content?
- Does any vendor in the chain train on my data? How do you know?
- If my data is regulated, what agreements and reviews do we need?
About cost and upkeep
- What will model usage cost per job, and show me the math?
- Who pays the model and hosting bills, and is there any markup?
- What happens when a model is retired or a price changes?
- What does ongoing maintenance include, and how much does it cost?
- Can another developer take this over if you leave?
A good developer welcomes these questions. Vague answers about "proprietary methods" should make you cautious. So should answers that cite a benchmark position and nothing else.
How to read a benchmark claim
Candidates may tell you they use "the best model." Ask them how they know. Benchmarks measure general tasks, not your job, and positions change often.
See the current AI agent leaderboard for where models stand this week and the methodology page for how positions are combined. A strong candidate treats those as a way to shortlist models, then tests on your own cases. A weak candidate treats them as the answer.
Red flags
- No evaluation plan. Testing is "we will try it out and see."
- Only demos. They can show a video but no production system or logs.
- Guaranteed results. Anyone promising a specific accuracy or savings figure before seeing your data is guessing.
- Full autonomy on day one. There is no approval step for money, customers, or legal actions.
- Broad credentials. The agent gets an admin account because it is easier.
- No answer on prompt injection. Or they say their model handles that.
- Vendor lock-in by design. You will not own the prompts, code, or test cases.
- Agent washing. Gartner warns of vendors rebranding chatbots and automation as agents. It estimates only about 130 of the thousands of agentic AI vendors are real, and consultants can do the same.
- Pressure to go broad. They propose ten agents before one works.
- Unclear AI costs. Model fees are "included" with no cap or explanation.
What it costs to hire
We do not publish a rate range, because we could not find a neutral source that supports one. Two reference points can help. The US Bureau of Labor Statistics reports median pay of $135,980 a year for software developers as of May 2025.
Freelance marketplaces show how widely offered rates vary. Upwork's AI developer directory displayed individual profiles at rates as low as $10 and $20 an hour on the day we checked. Treat a low rate as a question to investigate, not a bargain, since the expensive parts of this work are testing and containment.
For a full picture of project costs, see our guide on how much it costs to build an AI agent. Compare the options to a staff role too, including benefits and management time.
Run a paid pilot
Never go straight to a large contract. Pay for a small project that proves the working relationship and produces something real.
A good pilot has these features:
- One narrow job with a clear end.
- Real data, or realistic sample data if the real data is sensitive.
- A written test set agreed up front.
- Human approval on every external action.
- A short report at the end: what worked, what failed, and the cost per job.
Score the result yourself using our agent evaluation scorecard. If they get defensive about failures, you have learned something useful. Every agent fails sometimes, and what matters is how clearly the developer can show you where.
Hiring checklist
- [ ] One-page brief written
- [ ] Existing products ruled out
- [ ] Three candidates asked the same questions
- [ ] A production system seen, with logs
- [ ] Reference call with an unprompted client
- [ ] Security answers reviewed, by an outside person if data is sensitive
- [ ] Paid pilot agreed, with written test cases
- [ ] Cost per job estimated, with the math shown
- [ ] Contract reviewed by your attorney
What the contract should cover
Have your attorney review the agreement. This is general information, not legal advice. Ask for clear terms on these points:
- Ownership: code, prompts, test cases, configuration, and logs.
- Accounts: model, hosting, and tool accounts should be in your name.
- Data handling: where data goes, retention, and deletion when the work ends.
- Acceptance: results measured against the agreed test set.
- Change control: how new scope is priced.
- Support: response times and what maintenance includes.
- Handover: documents good enough for another developer to continue.
- Liability: who is responsible for what when the agent makes a mistake.
For a framework your team can follow, NIST's AI Risk Management Framework is voluntary and free to read. It gives you shared language for risk discussions. Larger companies may want to ask candidates how their process lines up with it.
Regulated work adds requirements. Health, legal, tax, and financial outputs should go through a qualified professional, whichever developer you hire. Do not accept a promise that the agent makes that review unnecessary.
Where to go next
Decide which actions must stay under human control before you sign. If your project involves connecting tools, ask how the developer plans to use MCP, the open standard for connecting AI to tools. For a conversation about a specific brief, contact our team.
Frequently asked questions
Should I hire a freelancer or an agency?
A freelancer often suits one narrow agent with a clear owner on your side. An agency may suit several systems or a need for continued support. In either case, ask who will actually do the work and what happens if that person leaves.
Do they need machine learning experience?
Usually not in the sense of training models. Agent work is mostly software engineering, integration, security, and evaluation. Domain knowledge of your process may matter more than a research background.
How long should a pilot take?
Keep it short enough that a failure stays cheap, and long enough to run real cases. We cannot give a reliable standard length, because it depends on your integrations. Agree an end date and a written test set before work begins.
How do I check a developer's work if I'm not technical?
Judge outcomes rather than code. Ask for test results, logs of real runs, and a demonstration on cases you choose. For sensitive systems, pay an independent engineer for a short review before launch.
