Your agent doesn't need a better prompt. It needs an eval suite.
Prompt tweaking feels like progress because the last failure disappears. An eval suite is what tells you whether the other forty came back.
Almost every agent project we're asked to rescue has the same shape. The agent works in the demo. It works when the person who built it drives it. Then it reaches real users and starts doing things nobody can explain, and the team's response is to go back into the prompt and add another paragraph of instructions.
That loop can run for months. It feels productive — each edit makes the specific failure you just saw go away. What it cannot tell you is whether the edit broke four things that used to work, because nothing is measuring the things that used to work.
The cheapest useful eval is a spreadsheet
You do not need an evals platform to start. You need a list of inputs and what should happen for each. Thirty rows is enough to change how a project feels. Write them from the transcripts you already have: every time the agent does something wrong in production, that conversation becomes a row.
The important discipline is that a row records the expected behaviour, not the expected wording. Agents phrase things differently every run, and an eval that asserts on exact text will fail constantly for reasons you don't care about. Assert on what can be checked mechanically:
- Which tool was called, with which arguments
- Whether the agent refused, escalated, or answered
- Whether a specific fact appeared in the answer
- Whether it stayed inside a policy boundary — refund ceilings, date ranges, who it's allowed to talk about
Three of those four are exact assertions on structured data, not fuzzy judgement. Tool calls are the highest-signal thing to test because they're where an agent causes real-world effects, and they're perfectly checkable.
Separate retrieval failures from reasoning failures
When an agent gives a wrong answer, there are two very different root causes and they need different fixes. Either the right information never reached the model, or it did and the model reasoned badly. Teams that don't separate these end up rewriting prompts to fix what is actually a search problem.
Log the retrieved context alongside every eval result. Then you can ask the only question that matters first: was the answer present in what we gave it? If it wasn't, no prompt will fix it — that's a chunking, indexing, or query-rewriting problem. We've seen supposedly intractable 'hallucination' issues turn out to be a retriever returning the right document with the relevant paragraph cut off at a chunk boundary.
Confidence thresholds beat more instructions
The instinct when an agent gets something wrong is to tell it, in the prompt, to be more careful. That works less well than making not-answering a first-class outcome.
Give the agent an explicit escalation path and make the bar for using it concrete: no matching record found, confidence below the threshold, a value outside the policy range, more than one plausible interpretation of the request. Then measure the escalation rate as a metric you actually care about. An agent that escalates 30% of the time and is right on the other 70% is a system you can deploy. An agent that answers everything and is right 85% of the time is one you cannot, because you don't know which 15%.
The number to put in front of a stakeholder isn't accuracy. It's accuracy at a given escalation rate. One without the other is unfalsifiable.
Run the suite on every change, including model upgrades
Once the suite exists, wire it into CI the same way you would unit tests. The failure mode it protects against is subtle: someone adds a sentence to the system prompt to fix one customer complaint, and the agent's behaviour shifts on a class of requests nobody thought to re-check.
It also turns model upgrades from a leap of faith into a measurement. When a new model ships, you run the suite against it and read the diff. Without evals, upgrading means shipping and hoping; teams in that position tend to freeze on an old model for years because they have no way to gain confidence.
What good looks like
- A case file per known failure, added the day the failure is reported
- Assertions on tool calls and policy boundaries, not on prose
- Retrieved context logged next to every result, so retrieval and reasoning failures are distinguishable
- An explicit escalation path, with the escalation rate tracked as a headline metric
- The suite running in CI, blocking prompt and model changes that regress it
None of this is exotic infrastructure. It's the difference between an agent you can change with confidence and one everybody is quietly afraid to touch.