The Open-Source AI Stack
RSS
All models

Models · qwen3-5

Qwen3.5 35B-A3B

Open Alibaba · 2026-02-16 · Apache-2.0

A small Qwen3.5 mixture-of-experts: about 35B total with roughly 3B active per token, so it decodes very fast for its size. A pure-dense 35B at Q8 would be memory-bandwidth-bound near single-digit tokens/sec on a 256 GB/s box; the measured high speed on Strix Halo is itself evidence this SKU is sparse, not dense. It is typically run at Q8.

Architecture

tokens in Embedding vocab not disclosed · qwen tokenizer × 48 layers Grouped-Query Attention RoPE + YaRN context 262,144 tokens MoE Router ? experts total · ? active per token Output projection tokens out
Schema-generated from data/models.yaml. Every label is auditable against the model's sources.

Specs

Architecture
moe
Total params
35B
Active params
3B
Context window
262K tokens
Attention
gqa
Position encoding
rope-yarn
Post-training
sft, rlhf
OSI-approved
yes
Data released
no
Training code
not released

Available quantizations

GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8. runs on llama.cpp, Ollama
AWQ Activation-aware 4-bit weight quantization for GPU serving. runs on vLLM, SGLang
GPTQ Post-training 4-bit weight quantization for GPU serving. runs on vLLM, SGLang, Transformers
MLX Apple MLX 4/8-bit layout for Apple silicon. runs on Apple MLX
FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs. runs on vLLM, SGLang, TensorRT-LLM

Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.

Notable innovations

  • · Sparse MoE with about 3B active per token

Sources