How do you keep AI agents reliable on long tasks?
Agent reliability is a math problem before it is a prompt problem. A run that is 95 percent right at each step is only 59 percent right after 10 steps. What holds a long job together is shorter runs, a written progress file, a hard stop rule, and a human yes before anything irreversible.
Why long runs go wrong
An agent that gets each step right 95 percent of the time finishes a 10 step job only 59 percent of the time. The failures are rarely dramatic. The agent forgets a decision it made 20 minutes earlier, redoes work it already finished, or treats a rule from the top of its instructions as optional. Princeton's reliability tracker measures the gap directly: agents that can solve a task often fail to solve the same task on a repeat run, and that gap has barely moved in two years while accuracy scores kept climbing.
The controls that move the number
Rewording the instructions does not fix this. One Hacker News commenter described the current state of control as pleading with it not to do something. Four things work better:
- Shorter runs. Ten steps beat forty. Split a long job into pieces that each end in a checkable result.
- A written progress file the agent updates as it goes and rereads before every run, so nothing rides on its memory.
- A stop rule. After a set number of tries, the run stops and asks a person.
- Approval before anything irreversible. Sending an email, paying an invoice, deleting a record: those wait for a yes.
Where to put the checkpoints
Put a person at the points where a mistake costs money or cannot be undone, not at every step. If the work is reading and sorting (labeling invoices, drafting replies) let it run and review a sample afterwards. If it spends, sends, or deletes, keep the approval gate and accept the slower pace. A run that goes 30 minutes with nobody seeing anything is too long for work that touches customers.
Last updated: May 20, 2026