Glossary
structured output
Making a model emit text that conforms to a schema or type (often JSON matching a JSON Schema) rather than free-form prose, so downstream code can parse it reliably.
The practice of forcing a model’s output into a known shape instead of free-form prose. The caller supplies a JSON Schema (or a type, or a regular expression); the model returns text that parses against it. This is what makes a model’s output safe to feed into code: a parser never has to guess whether the model wrapped its answer in prose, used the wrong key, or emitted a trailing comma.
There are two ways to get there, and they are often combined. One is training: a model fine-tuned to emit JSON when asked usually does. The other is constrained decodingruntimeMasking the model's next-token probabilities at each step so only tokens allowed by a grammar or schema can be sampled, forcing the output to be structurally valid. Open full entry , where the runtime masks the next-token choices so only schema-valid tokens can be sampled. Training alone is best-effort; constrained decoding makes conformance a property of the decoder, not a hope about the weights. The weaker, older form is JSON moderuntimeA weaker predecessor to structured output: the model is constrained to emit syntactically valid JSON, but not necessarily JSON that conforms to a specific schema. Open full entry , which guarantees valid JSON but not a specific schema.
Structured output is the substrate under function callingagentsA pattern where a model emits a structured call (function name plus arguments), the runtime executes it, and the result returns as input on the model's next turn. Open full entry : a tool call is just a structured object (tool name plus arguments) the model is constrained to produce. It is also how you get reliable extraction, classification, and form-filling out of open models served by vLLMruntimeAn open-source inference engine introduced by UC Berkeley in 2023, built around PagedAttention to manage KV cache memory and serve tokens efficiently under load. Open full entry , SGLangruntimeAn open inference engine from the LMSYS team featuring RadixAttention for prefix sharing and a structured-generation frontend, particularly strong on agent and tool-calling workloads. Open full entry , or llama.cppruntimeGeorgi Gerganov's C++ inference engine optimized for CPUs and consumer GPUs, the on-device standard and the engine behind Ollama, LM Studio, and most local-first AI products. Open full entry , all of which expose a schema or grammarruntimeA formal description of allowed output structure (for example llama.cpp's GBNF) that a decoder is constrained to; a JSON Schema can be compiled into one. Open full entry parameter on the request.