Reliable AI products do not treat model text as an instruction to blindly execute. They turn a model response into a constrained, inspectable interface: structured data for the application, tools with explicit arguments, server-side validation, and a clearly bounded retry path.
Prefer a schema over prose parsing
Asking for “valid JSON” helps, but it is not a guarantee. Define the fields, allowed values, and types that your application needs, then validate every result after it arrives. If the result fails validation, do not let it leak into the next system. Either ask for a narrowly specified repair, use a fallback route, or return an actionable error.
{
"action": "create_ticket",
"priority": "low | normal | urgent",
"summary": "string, max 160 chars"
}Tool calls are proposals, not permissions
Tool calling allows a model to request an application function with arguments. The server should validate those arguments, confirm the user is authorized, enforce business rules, and only then execute. A model must never be the source of authority for a payment, data deletion, permission change, or external message.
Make side effects idempotent
Networks fail and clients retry. If a tool can create an order, send an email, or change a record, include an idempotency key and store execution state. The same call should be safe to receive twice. This turns a temporary timeout from a possible duplicate action into a recoverable event.
Use a bounded repair loop
- Validate schema and business constraints.
- If invalid, provide the model only the validation errors and require a corrected structure.
- Cap the repair attempts—usually one or two are enough.
- Log failure class, model route, and prompt version.
- Escalate or fail clearly when the cap is reached.
Evaluate the whole workflow
Test malformed inputs, missing fields, conflicting instructions, unavailable tools, duplicate calls, and unsafe arguments. A model with excellent prose can still be a poor fit if its tool calls are unreliable. Measure schema validity, correct-tool selection, safe rejection, and successful completion—not just answer quality.