Structured Outputs: Getting Reliable JSON from LLMs
Most production AI features do not want prose. They want data: a category, extracted fields, a decision. Getting that reliably is an engineering problem with well-known solutions.
- Published
Use schemas, not instructions
"Reply in JSON" in a prompt works most of the time. Providers that support structured outputs let you pass a JSON schema and guarantee the output conforms to it. Use that wherever available; it removes a whole class of parsing failures.
Design schemas the model fills well
- Use enums for categories — the model picks from your list instead of inventing labels.
- Allow null or "unknown" for fields that may be absent, so the model is not forced to guess.
- Add a short reasoning or evidence field before the decision field when accuracy matters.
- Keep nesting shallow and field names descriptive.
Validate anyway
- Validate against the schema in code.
- Apply business rules: dates in range, totals consistent, IDs that exist.
- On failure, retry once with the validation error included.
- On repeated failure, route to a person — never silently accept.
Tool calls are structured outputs too
Tool arguments follow the same rules. A well-designed tool schema with enums and clear descriptions produces more accurate calls than a free-form parameter.
Frequently asked questions
Do structured outputs reduce quality?
Not noticeably for well-designed schemas. Overly complex schemas can hurt — keep them simple.
What about very long extractions?
Split into sections or pages and merge results in code.
Can small models do this?
Often yes, for well-defined extraction. Test on your data with evals.
