Architecture
data/models.yaml. Every label is auditable
against the model's sources.
Specs
- Architecture
- moe
- Total params
- 1T
- Active params
- 32B
- Experts
- 384 total · 8 active
- Context window
- 128K tokens
- Attention
- mla
- Position encoding
- rope
- Pretraining tokens
- 15.5T
- Post-training
- sft, rlhf
- OSI-approved
- no
- Data released
- no
- Training code
- not released
Benchmarks
Each score carries the date it was published; we never infer or interpolate missing scores.
General reasoning
| MMLU-Pro | 81.1 | as of 2025-07-15 | source ↗ |
| GPQA-Diamond | 75.1 | as of 2025-07-15 | source ↗ |
Code
| SWE-Bench Verified | 65.8 | as of 2025-07-15 | source ↗ |
Recommended use cases
- agentic tool use
- code agent backend
- long-context tasks
Available quantizations
GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8.
runs on llama.cpp, Ollama
FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs.
runs on vLLM, SGLang, TensorRT-LLM
Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.
Notable innovations
- · 1T-param open-weights MoE
- · Agentic post-training focus
Lineage
Moonshot's first widely-released open-weights MoE.