Llama 3.1 8B Instruct

Cost

— / Mtok input

— / Mtok output

Together AI · as of 2026-05-19

via Artificial Analysis ↗

Speed

— tok/sec output

Together AI · as of 2026-05-19

via Artificial Analysis ↗

Architecture

Schema-generated from data/models.yaml. Every label is auditable against the model's sources.

Specs

Architecture: dense
Total params: 8.0B
Active params: 8.0B
Context window: 131K tokens
Attention: gqa
Position encoding: rope-llama3
Pretraining tokens: 15.0T
Training hardware: H100
Post-training: sft, dpo
OSI-approved: no
Data released: no
Training code: not released

Benchmarks

Each score carries the date it was published; we never infer or interpolate missing scores.

General reasoning

GPQA-Diamond

30.4

as of 2024-07-23

source ↗

Recommended use cases

long-context tasks at the 8B class
RAG
tool use

Available quantizations

GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8. runs on llama.cpp, Ollama

AWQ Activation-aware 4-bit weight quantization for GPU serving. runs on vLLM, SGLang

GPTQ Post-training 4-bit weight quantization for GPU serving. runs on vLLM, SGLang, Transformers

EXL2 ExLlamaV2's variable-bitrate format for consumer GPUs. runs on ExLlamaV2

MLX Apple MLX 4/8-bit layout for Apple silicon. runs on Apple MLX

FP8 8-bit float, frequently a native release on Hopper / Blackwell GPUs. runs on vLLM, SGLang, TensorRT-LLM

bitsandbytes On-the-fly NF4 / INT8 weight quantization inside Transformers. runs on Transformers

Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.

Notable innovations

· 128K context via continued pretraining
· Tool-use post-training

Lineage

Continued pretraining from the Llama 3 8B base, separate from the Llama 3 8B Instruct line.