Where agents actually fail once real people use them
Read this before you sign off a pilot. Every item here is something that works in testing and breaks in production.
- 01
It is confidently wrong rather than unsure.
A model's fluency is unrelated to its accuracy, so the wrong answers arrive in the same calm tone as the right ones. Users learn to trust it during the easy weeks and then get caught. The fix is citations and a visible confidence boundary, designed in from the start.
- 02
The demo questions were the easy ones.
Whoever built the demo unconsciously asked questions it could answer. Real users arrive with two questions at once, a typo, an attachment, and context from an email you cannot see.
- 03
It cannot say 'I don't know'.
in the PDF
- 04
Someone finds the edge of the guardrails and pushes.
in the PDF
- 05
It succeeds and nobody notices the side effect.
in the PDF
- 06
Latency makes it useless even though it is correct.
in the PDF
- 07
It quietly degrades after a provider update.
in the PDF
5 more checks in the PDF
The complete checklist — every check with the reasoning under it, the closing note, and the other 12 documents — comes as one PDF.