The Open-Source AI Stack
All models

Models · mistral

Mistral Nemo 12B Instruct

Open Mistral AI · 2024-07-18 · Apache-2.0

Co-developed with NVIDIA and released in July 2024 under Apache 2.0, Nemo positioned itself as a drop-in upgrade to Mistral 7B with a 128K context and quantisation-aware training that enables lossless FP8 inference. Ships with a new Tekken tokenizer trained on 100+ languages that is roughly 30% more efficient than Llama 3's tokenizer on code and major non-English languages.

Cost

— / Mtok input
— / Mtok output

Mistral La Plateforme · as of 2026-05-19

via Artificial Analysis ↗

Architecture

tokens in Embedding vocab 131,072 · tekken tokenizer × 40 layers Grouped-Query Attention RoPE context 131,072 tokens Dense MLP SwiGLU activation (standard) 12B active params Output projection tokens out
Schema-generated from data/models.yaml. Every label is auditable against the model's sources.

Specs

Architecture
dense
Total params
12B
Active params
12B
Context window
131K tokens
Attention
gqa
Position encoding
rope
Post-training
sft, dpo
OSI-approved
yes
Data released
no
Training code
not released
QAT
fp8 (official)

Benchmarks

Each score carries the date it was published; we never infer or interpolate missing scores.

General reasoning

MMLU 68.0 as of 2024-07-18 source ↗

Available quantizations

QAT fp8 Trained to be quantized: quantization-aware training, with official checkpoints (2024-07-18).

Trained with quantisation awareness for lossless FP8 inference. FP8-targeted, not int4.

source ↗
GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8. runs on llama.cpp, Ollama
AWQ Activation-aware 4-bit weight quantization for GPU serving. runs on vLLM, SGLang
EXL2 ExLlamaV2's variable-bitrate format for consumer GPUs. runs on ExLlamaV2
MLX Apple MLX 4/8-bit layout for Apple silicon. runs on Apple MLX
FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs. runs on vLLM, SGLang, TensorRT-LLM
bitsandbytes On-the-fly NF4 / INT8 weight quantization inside Transformers. runs on Transformers

Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.

Notable innovations

  • · Tekken tokenizer with ~128K vocab and improved multilingual compression
  • · Quantisation-aware training for lossless FP8 inference
  • · Drop-in replacement for Mistral 7B at 128K context

Sources