Structured Output for SRE Agents: When to Use It

JSON schemas force the model to return parseable output but cost reasoning quality. The cases where structured output is worth it, and the cases where free-form plus a parser wins.

When structured output is the right choice

Structured output works best when the consumer is code. If the agent's output goes to a tool that needs to dispatch on a field, structured output is non-negotiable. "severity" must be a string from a fixed set; "score" must be a float.

It also works when the agent's output is a contract between agents. The triage agent emits a hypothesis with named fields; the remediation agent reads the hypothesis. Structure is the contract.

Structured output is not always best for the human-facing case. A free-form summary is often more useful for an on-call engineer than a JSON object. Match the format to the consumer.

Schema design that holds up

Keep schemas small. Five to ten fields is plenty for most agent outputs. Larger schemas confuse the model; smaller schemas force ambiguity.

Use enums where possible. Severity should be an enum (low/medium/high/critical), not a free-form string. Enums prevent drift and make downstream code simpler.

Include a confidence field. The agent should self-report how confident it is in each output. Downstream code can branch on confidence: high-confidence auto-routes, low-confidence escalates.

The reasoning quality cost

Forcing a JSON schema costs reasoning quality. The model spends some attention on schema compliance instead of the underlying problem. The penalty is real but small for modern models; budget 5-10% accuracy.

For trivial schemas (one or two fields), the cost is negligible. For complex schemas (deeply nested, with conditional fields), the cost can hit 20%. Keep schemas flat.

If the cost is unacceptable, switch to free-form output and parse it with a second model call. The two-call pattern is more expensive in dollars but recovers the reasoning quality.

Validation as a safety net

Every structured output goes through validation before being trusted. The model can produce malformed JSON, miss required fields, or invent enum values. Validation catches all three.

When validation fails, retry once with the error message in the prompt. The model usually self-corrects on the second try. After two failures, fall back to a manual path.

Track validation failure rates. A spike usually indicates a model regression or a schema drift. The metric is a leading indicator of agent health.

Evolving the schema over time

Schemas drift. Fields get added, removed, renamed. Treat the schema as an API; version it. v1 is the launch contract; v2 is the next.

Migration is non-trivial. Existing prompts, tools, and downstream code all reference the schema. Plan migrations as you would plan database migrations: explicit, reversible, dual-write where possible.

Resist the temptation to over-evolve. Schema churn confuses the model and the team. Only add a field when there is a concrete consumer for it.

What to do this week

Pick one agent that emits free-form output today. Sketch a structured-output schema for it: five to seven fields, mostly enums. A/B run for a week (free-form vs structured). Compare reasoning quality, downstream parsing reliability, and overall pipeline simplicity. Most agents that interact with code benefit from the switch.