The Open-Source AI Stack
RSS
All models

Models · qwen3

Qwen3 Next 80B-A3B Instruct

Open Alibaba · — · Apache-2.0

Preview of a new ultra-sparse Qwen architecture: 80B total, 3B active per token (3.75 percent of params), 512 experts with 10 routed plus 1 shared. Hybrid layout alternates Gated DeltaNet and gated attention. Alibaba reported 10 percent of the training cost of Qwen3-32B and 10x the inference throughput beyond 32K context.

Architecture

tokens in Embedding vocab not disclosed · qwen tokenizer × 48 layers Attention (not disclosed) RoPE + YaRN context 262,144 tokens MoE Router 512 experts total · 10 active per token shown: 32 of 512 Output projection tokens out
Schema-generated from data/models.yaml. Every label is auditable against the model's sources.

Specs

Architecture
moe
Total params
80B
Active params
3B
Experts
512 total · 10 active
Context window
262K tokens
Attention
hybrid-gated-deltanet
Position encoding
rope-yarn
Pretraining tokens
15.0T
Post-training
sft, rlhf
OSI-approved
yes
Data released
no
Training code
not released

Benchmarks

Each score carries the date it was published; we never infer or interpolate missing scores.

General reasoning

MMLU-Pro 80.6 as of 2025-09-11 source ↗

Code

LiveCodeBench 56.6 as of 2025-09-11 source ↗

Math

AIME 2025 69.5 as of 2025-09-11 source ↗

Held-out / arena

IFEval 87.6 as of 2025-09-11 source ↗

Available quantizations

GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8. runs on llama.cpp, Ollama
AWQ Activation-aware 4-bit weight quantization for GPU serving. runs on vLLM, SGLang
GPTQ Post-training 4-bit weight quantization for GPU serving. runs on vLLM, SGLang, Transformers
EXL2 ExLlamaV2's variable-bitrate format for consumer GPUs. runs on ExLlamaV2
MLX Apple MLX 4/8-bit layout for Apple silicon. runs on Apple MLX
FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs. runs on vLLM, SGLang, TensorRT-LLM
bitsandbytes On-the-fly NF4 / INT8 weight quantization inside Transformers. runs on Transformers

Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.

Notable innovations

  • · Hybrid Gated DeltaNet + gated-attention layout
  • · 3.75 percent active-param fraction (80B / 3B)
  • · 1M-token extensibility via YaRN
  • · 10x inference throughput beyond 32K vs Qwen3-32B per Alibaba

Lineage

New ultra-sparse Qwen architecture preview.

Derived from

Qwen 3 235B A22B Instruct 2025-04-28

Sources