Action hallucination in AI agents: what causes it and how we fixed it
Agents that invent tool calls are the most dangerous failure mode in production. Here is the root cause and the guardrail pattern that eliminated it for us.
Action hallucination is the failure mode where an AI agent invents a tool call that does not exist, calls a real tool with fabricated parameters, or claims it completed an action it never actually performed. It is more dangerous than a wrong answer, because a wrong answer is visible — a fabricated action silently corrupts state downstream.
We hit this in production while building an agentic operations workflow. The agent would, occasionally, report that it had "updated the record" when no update call had ever been made. The language model had learned that confirming an update was the expected completion of the pattern — so it completed the pattern in text without completing it in fact.
The root cause
The cause is structural, not a model defect. An agent that decides what to do and narrates what it did in the same generation has no separation between intent and outcome. The model is rewarded for producing a coherent, complete-looking transcript, and "I called the tool and it succeeded" is more coherent than "I attempted the call and it failed." Left unconstrained, it optimizes for the tidy story.
Prompting harder does not fix this. Telling the model "never claim an action you did not take" reduces the frequency but does not eliminate it, because you are asking the same system that hallucinates to also police its own hallucination.
The guardrail pattern that eliminated it
The fix is to make the agent incapable of narrating an action that did not happen. We separated deciding from reporting. The model proposes a structured action — a tool name and validated parameters — and that proposal is executed by deterministic code, not by the model. Only after the tool returns a real result does the system generate any user-facing confirmation, and that confirmation is grounded in the actual tool response, not the model's imagination.
Concretely: every tool call is schema-validated before execution, every execution returns a typed result, and every confirmation message is templated from that result. If the tool fails, the agent must say so, because the failure is part of the result it is given. We also log every proposed action against every executed action, so any divergence is caught in review even if it never reaches a user.
After shipping this separation, fabricated-action reports dropped to zero in our logs. The agent still makes mistakes — but it can no longer lie about having acted, because it is structurally unable to describe an action the system did not perform.