The Open-Source AI Stack
All models

Models · gemma

Gemma 3 27B IT

Source-available Google DeepMind · 2025-03-12 · Gemma Terms of Use

Gemma 3 27B IT introduced native multimodal input (text plus images at 896x896 normalized to 256 tokens) and pushed context to 128K at the 27B scale. Trained on 14T tokens across 140+ languages on TPU v5p. The Gemma Terms of Use remain source-available rather than OSI-approved.

Architecture

tokens in Embedding vocab not disclosed · gemma tokenizer × N layers GQA + Sliding-window (interleaved) RoPE context 131,072 tokens Dense MLP SwiGLU activation (standard) 27B active params Output projection tokens out
Schema-generated from data/models.yaml. Every label is auditable against the model's sources.

Specs

Architecture
dense
Total params
27B
Active params
27B
Context window
131K tokens
Attention
hybrid-gqa-sliding
Position encoding
rope
Pretraining tokens
14.0T
Training hardware
TPU v5p
Post-training
sft, rlhf
OSI-approved
no
Data released
no
Training code
not released
QAT
int4 (official)

Benchmarks

Each score carries the date it was published; we never infer or interpolate missing scores.

General reasoning

MMLU-Pro 67.5 as of 2025-03-12 source ↗

Code

HumanEval 87.8 as of 2025-03-12 source ↗

Recommended use cases

  • multimodal text-and-image instruct
  • long-context document tasks
  • multilingual deployment

Available quantizations

QAT int4 Trained to be quantized: quantization-aware training, with official checkpoints (2025-04-18).

Official int4 QAT checkpoints (about 5,000 QAT steps matching the BF16 logits). 27B drops from 54GB BF16 to 14.1GB int4, fitting a 24GB RTX 3090.

source ↗
GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8. runs on llama.cpp, Ollama
AWQ Activation-aware 4-bit weight quantization for GPU serving. runs on vLLM, SGLang
GPTQ Post-training 4-bit weight quantization for GPU serving. runs on vLLM, SGLang, Transformers
EXL2 ExLlamaV2's variable-bitrate format for consumer GPUs. runs on ExLlamaV2
MLX Apple MLX 4/8-bit layout for Apple silicon. runs on Apple MLX
FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs. runs on vLLM, SGLang, TensorRT-LLM
bitsandbytes On-the-fly NF4 / INT8 weight quantization inside Transformers. runs on Transformers

Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.

Notable innovations

  • · Native multimodal at 27B scale
  • · 128K context with hybrid sliding-window attention
  • · 140+ language coverage

Known limitations

  • · Gemma Terms of Use are source-available, not OSI-approved. source ↗

Lineage

Successor to Gemma 2 27B IT. Siblings at 1B, 4B, and 12B in the Gemma 3 family.

Sources