Structured Outputs Beyond JSON Mode, Schemas as the Contract
Pull-quote: “JSON mode promises you a parse. A schema promises you a shape. Neither promises you the truth, which is why the contract needs three layers, not one.”
Why this matters
Constrained decoding solved the problem everyone complained about. Modern APIs can guarantee that output parses as JSON and, with schema-enforced modes, that it matches a declared structure. The regex era of scraping fields out of prose is over. What replaced it is a subtler failure: teams treat “it validates” as “it is correct,” and ship model outputs into live systems on the strength of a syntax check. A response can conform to the schema perfectly and still put a plausible value in a field the model had no evidence for. The schema is not a checkbox on an API call. It is the contract between a probabilistic component and the deterministic system consuming it, and contracts deserve engineering.
Three guarantees, three layers
| Layer | What it guarantees | What enforces it |
|---|---|---|
| Syntax | Output parses as JSON | Constrained decoding |
| Shape | Fields, types, enums match the schema | Schema-enforced mode, validator |
| Semantics | Values are consistent and evidenced | Your code: range checks, cross-field rules, provenance |
The first two layers now come from the platform. The third never will, because the platform does not know that an end date before a start date is impossible, that a confidence of 0.99 on a five-word input is suspicious, or that a part number must exist in your catalog. Semantic validation is ordinary code, it is cheap to write against typed objects, and it is the layer that actually protects the system downstream.
The contract lifecycle
schema (versioned, owned)
│
├──► prompt: schema + field descriptions ──► model
│ │
├──► constrained decode (syntax, shape) ◄──────┘
│ │
├──► semantic validators (ranges, cross-field, catalog)
│ │ fail
│ ├──────► retry with validator errors in context
│ │ fail again
│ └──────► escalate: larger model or human queue
▼
typed object ──► downstream system, event bus, databaseTreat the schema like an API contract, because that is what it is:
- Version it in git, next to the prompt that references it. A field rename is a breaking change for every consumer; it should look like one in review.
- Generate types from it. One schema, one source of truth, typed objects in every consumer. Divergence between what the model is asked for and what the code expects is how quiet corruption starts.
- Feed validator errors back before escalating. A retry that includes the specific violation fixes most failures for one extra call. Escalation to a larger model or a human queue is the documented last resort, not the default.
- Log conformance and repair rates. A rising repair rate is an early signal that the task, the prompt, or the model changed under you.
Design the schema for honesty
The shape of the schema decides whether the model can tell you the truth. Closed enums beat free text for anything a downstream system branches on. Every extracted field needs a representable “unknown,” because a schema with only required, non-nullable fields is an instruction to invent. Pair values with evidence fields, a source span or a reason code, so the claim carries its own audit trail. And keep one schema per decision: a mega-schema that captures everything invites the model to fill the parts it should have left empty.
Where the contract earns its keep
The value is clearest in pipelines where model output feeds other systems rather than a reader. An anomaly-detection pipeline that moves findings as typed events lets every consumer, from scoring to analyst narrative, read the same contract rather than parse prose. An evidence-screening workflow that returns each call as a structured verdict with per-criterion reason codes is what makes thousands of include-exclude decisions auditable in a PRISMA 2020 review instead of buried in paragraphs. In both cases the schema is doing governance work, not just formatting work.
Closing
Structured outputs ended the parsing problem, not the correctness problem. The teams that ship reliable systems treat the schema as a versioned, owned contract; validate semantics in code the platform cannot know; and design fields so the model can say “unknown” instead of inventing. Syntax is free now. Meaning is still yours to enforce.