A tool demo looks magical: the model selects a function, generates arguments, receives a result, and completes the task.
Production adds expired credentials, partial responses, rate limits, stale data, invalid arguments, duplicate requests, and tools that succeed after the user has cancelled.
Treat Tools as Untrusted Boundaries
Validate every argument. Enforce authorization outside the model. Set timeouts, return typed errors, and make side effects idempotent where possible. The model should never turn a natural-language request into unchecked authority.
Tool errors should be designed for two audiences. Code needs a stable category. The model needs enough explanation to recover without seeing secrets or internal stack traces.
Decide Who Can Retry
Automatic retry is appropriate for some transient failures. It is dangerous for payments, messages, deletion, and other side effects. Give each tool a retry policy and require confirmation for consequential actions.
The agent also needs a stopping rule. Repeating the same failed call with slightly different wording is not recovery.
Evaluate the Path
Do not judge only the final answer. Check whether the agent:
- selected the correct tool;
- passed valid arguments;
- respected permissions;
- interpreted the result correctly;
- avoided unnecessary calls;
- and responded appropriately to failure.
A correct answer reached through an unsafe sequence should still fail.
Reliable tool calling comes from the system around the model: contracts, permissions, observability, idempotency, and evaluations that inspect the full trace.