grammar

A formal specification of what an output is allowed to look like, written in a notation a decoder can enforce. The best-known concrete form in the open ecosystem is GBNF, llama.cppⓘllama.cppruntimeGeorgi Gerganov's C++ inference engine optimized for CPUs and consumer GPUs, the on-device standard and the engine behind Ollama, LM Studio, and most local-first AI products. Chat about this → Open full entry →’s variant of Backus-Naur Form: a small set of rules that say, for example, an object opens with a brace, then key-value pairs, then a closing brace, and nothing else. The decoder reads the grammar to know which tokens are legal at each position.

Grammars and schemas are two views of the same constraint. A JSON Schema describes a valid document; a grammar describes the valid token sequences that produce one. Most engines accept a JSON Schema and compile it into a grammar internally, so authors rarely write GBNF by hand, but the grammar is what the constrained decoderⓘconstrained decodingruntimeMasking the model's next-token probabilities at each step so only tokens allowed by a grammar or schema can be sampled, forcing the output to be structurally valid. Chat about this → Open full entry → actually walks. Grammars also cover shapes JSON Schema cannot express directly, such as a specific regular-expression format or a restricted arithmetic expression.

Grammars are the lowest layer of the structured outputⓘstructured outputruntimeMaking a model emit text that conforms to a schema or type (often JSON matching a JSON Schema) rather than free-form prose, so downstream code can parse it reliably. Chat about this → Open full entry → stack. When a request to llama.cpp carries a grammar field, or a request to a server carries a JSON Schema, the result is the same: token sampling is restricted to what the grammar admits.