The Open-Source AI Stack
RSS
All models

Models · llama

Llama 3 8B Instruct

Source-available Meta · 2024-04-18 · Llama 3 Community License

The 15T-token pretrain and a 4x larger 128K-token vocab were the headline changes. The 8B class became the new local-deployment default, displacing Mistral 7B.

Cost

— / Mtok input
— / Mtok output

Together AI · as of 2026-05-19

via Artificial Analysis ↗

Speed

— tok/sec output

Together AI · as of 2026-05-19

via Artificial Analysis ↗

Architecture

tokens in Embedding vocab 128,256 · llama3 tokenizer × N layers Grouped-Query Attention RoPE (Llama 3 scaling) context 8,192 tokens Dense MLP SwiGLU activation (standard) 8.0B active params Output projection tokens out
Schema-generated from data/models.yaml. Every label is auditable against the model's sources.

Specs

Architecture
dense
Total params
8B
Active params
8.0B
Context window
8K tokens
Attention
gqa
Position encoding
rope-llama3
Pretraining tokens
15.0T
Training hardware
H100
Post-training
sft, dpo, rejection-sampling
OSI-approved
no
Data released
no
Training code
not released

Benchmarks

Each score carries the date it was published; we never infer or interpolate missing scores.

Recommended use cases

  • local deployment
  • edge inference
  • fine-tuning base

Available quantizations

GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8. runs on llama.cpp, Ollama
AWQ Activation-aware 4-bit weight quantization for GPU serving. runs on vLLM, SGLang
GPTQ Post-training 4-bit weight quantization for GPU serving. runs on vLLM, SGLang, Transformers
EXL2 ExLlamaV2's variable-bitrate format for consumer GPUs. runs on ExLlamaV2
MLX Apple MLX 4/8-bit layout for Apple silicon. runs on Apple MLX
FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs. runs on vLLM, SGLang, TensorRT-LLM
bitsandbytes On-the-fly NF4 / INT8 weight quantization inside Transformers. runs on Transformers

Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.

Notable innovations

  • · 15T-token pretrain
  • · 128K vocab

Lineage

Same architecture, 15T-token pretrain replaces Llama 2's 2T.

Derived from

Llama 2 70B Chat 2023-07-18

Derivatives

Sources