Architecture
data/models.yaml. Every label is auditable
against the model's sources.
Specs
- Architecture
- moe
- Total params
- 1.6T
- Active params
- 49B
- Context window
- 1.0M tokens
- Attention
- hybrid-csa-hca
- Position encoding
- unknown
- Pretraining tokens
- 32.0T
- Post-training
- sft, grpo, distillation
- OSI-approved
- yes
- Data released
- no
- Training code
- not released
Benchmarks
Each score carries the date it was published; we never infer or interpolate missing scores.
Available quantizations
GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8.
runs on llama.cpp, Ollama
FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs.
runs on vLLM, SGLang, TensorRT-LLM
Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.
Notable innovations
- · Hybrid CSA + HCA attention
- · Manifold-Constrained Hyper-Connections (mHC)
- · Muon optimizer at this scale
- · FP4 + FP8 mixed-precision MoE training
- · 1M context as default
Lineage
First V4 flagship; new hybrid attention and FP4 training stack.
Derived from
DeepSeek-V3.2 2025-12-01