The errors that teach

Our in-app assistant used to get stuck. Not confused: stuck. It would call the same tool with slightly different arguments, four or five times, then announce something it had not done. The instinct is to blame the model. That was wrong.

Descriptions warn, errors teach

The first fix anyone reaches for is the tool description. Ours grew scar tissue that way. Every incident added a sentence, until the search tool carried two hundred words containing a warning about a specific folder in a specific project. That text was paid for on every single call, and by the third turn of a conversation a small model has stopped reading it.

Meanwhile the moment where help actually lands, the failed call, returned this:

error: no such column: name

Which teaches nothing. The model guesses another column, and the guess varies, so a duplicate-call detector never catches it.

Now the same failure returns the real schema, the nearest actual columns, and a literal retry to copy. The correction arrives where the decision is being made, in context, at the exact moment it is needed. That is the whole trick. A weak model is bad at recalling a rule from forty turns ago and good at continuing a pattern in front of it, so the fix is to put the pattern in front of it.

What the reference implementations actually do

We read OpenAI's Codex to see how a serious harness handles this. The expectation was loop guards. There are none. No maximum turn count, no duplicate-call detector, no stall breaker. The backstops are context compaction and two prompt-level audits: do not declare yourself blocked until the same blocker has failed three times, and treat completion as unproven until you have checked it against real state.

It gets away with that because its tools are Read, Grep, and a shell over a filesystem, which is territory the model has seen a million times. Our tools are a private API for narrative analysis. Nobody has priors on our schema. So we keep the runtime guards Codex skipped, and we adopted the audits it has that we lacked.

Fewer tools, not more warnings

The last piece was subtraction. One catalog was feeding both the in-app chat and the plugin, so a small fast model was handed every tool including analysis internals and project lifecycle. Now each surface advertises only what it needs. A model cannot misroute to a tool it never saw.

Descriptions went on a diet afterwards, not before. The scar tissue was load bearing until something better replaced it.