XGrammar, from the MLC team, is an engine for grammar-based constrained decoding focused on being fast enough to run inside a production serving loop. It compiles a JSON Schema or context-free grammar and applies the per-step token mask that keeps generation valid, with an implementation aimed at keeping that masking cheap relative to the model forward pass. Its place in the ecosystem is as a backend rather than a user-facing API. Open serving engines integrate XGrammar (and alternatives like Outlines and llguidance) as the component that enforces structure, so a request carrying a JSON Schema is handed to one of these engines under the hood. The work is described in the XGrammar paper. For open-source AI it is part of why structured output is now a standard request parameter on open engines rather than a bolt-on: the masking is fast enough that schema-constrained generation is practical at serving scale. Teams rarely call it directly; they get it through the engine they already run.
The Stack · Runtime · Open source
XGrammar
Fast grammar-based constrained-decoding engine, used as a structured-output backend by several open serving engines.
Sources
- XGrammar on GitHub https://github.com/mlc-ai/xgrammar
- XGrammar: Flexible and Efficient Structured Generation (Dong et al., 2024) https://arxiv.org/abs/2411.15100
Want a follow-up? Ask the chat about XGrammar in context. It will compare to siblings at the same layer and ground every claim in the wiki.
Other projects at the Runtime layer
12 siblings · ordered open first
- vLLM Open source
Dominant open production inference engine; PagedAttention and continuous batching; NVIDIA / AMD / Intel / TPU support.
- SGLang Open source
RadixAttention plus structured generation; from the LMSYS team; gains for shared-prefix and agent workloads.
- llama.cpp Open source
Georgi Gerganov's local-first inference engine; defines the GGUF format; the on-device standard.
- Ollama Open source
Local model runner; Docker-style UX over llama.cpp; the easiest way to run open weights on your machine.
- Text Generation Inference (TGI) Open source
HuggingFace's production inference server; maintenance mode in 2026 as vLLM became the standard.
- MLC-LLM Open source
Cross-platform compilation (TVM-based); the 'LLM in your browser' or 'on your phone' standard.
- Outlines Open source
Structured-output library using finite-state-machine guided decoding to force model output to match a regex, JSON Schema, or grammar.
- llguidance Open source
Low-level Rust constrained-decoding engine that enforces grammars and JSON Schema at high throughput; powers Guidance.
- Guidance Open source
A programming model for constrained generation that interleaves control flow with model calls and enforces regex, grammars, and JSON Schema.
- Instructor Open source
Library for getting Pydantic-typed structured output from LLMs across many providers, built on function calling and JSON mode with automatic retries.
- LM Format Enforcer Open source
Token-filtering library that enforces a JSON Schema or regex during generation; integrates with Transformers, vLLM, and llama.cpp.
- TensorRT-LLM Source available
NVIDIA's closed-runtime counterpart; fastest on NVIDIA hardware; depends on closed CUDA kernels and the proprietary TensorRT compiler.