Architecture
data/models.yaml. Every label is auditable
against the model's sources.
Specs
- Architecture
- dense
- Total params
- 253B
- Active params
- 253B
- Context window
- 131K tokens
- Attention
- skip-attention
- Position encoding
- rope
- Pretraining tokens
- 65B
- Training hardware
- H100
- Post-training
- sft, grpo
- OSI-approved
- no
- Data released
- yes
- Training code
- not released
Benchmarks
Each score carries the date it was published; we never infer or interpolate missing scores.
General reasoning
| GPQA-Diamond | 76.0 | as of 2025-04-11 | source ↗ |
Code
| LiveCodeBench | 66.3 | as of 2025-04-11 | source ↗ |
Held-out / arena
| IFEval | 88.8 | as of 2025-04-11 | source ↗ |
Available quantizations
GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8.
runs on llama.cpp, Ollama
FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs.
runs on vLLM, SGLang, TensorRT-LLM
Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.
Notable innovations
- · 405B distilled to 253B via NAS
- · FFN fusion alongside skip attention
- · Single-node 8x H100 inference
Lineage
NAS-distilled Llama 3.1 405B at 253B for enterprise single-node deployment.