The Open-Source AI Stack
RSS
All models

Models · deepseek

DeepSeek-Coder-V2 Instruct

Open weights DeepSeek · 2024-06-17 · DeepSeek License

Code specialist branch of the V2 line. Continued pretraining from an intermediate V2 checkpoint on an additional 6 trillion code and math tokens, expanding programming language coverage from 86 to 338 and extending context from 16K to 128K. The Instruct variant reported HumanEval scores competitive with GPT-4 Turbo.

Architecture

tokens in Embedding vocab not disclosed · deepseek tokenizer × N layers Multi-head Latent Attention RoPE + YaRN context 128,000 tokens MoE Router 160 experts total · 6 active per token shown: 32 of 160 Output projection tokens out
Schema-generated from data/models.yaml. Every label is auditable against the model's sources.

Specs

Architecture
moe
Total params
236B
Active params
21B
Experts
160 total · 6 active
Context window
128K tokens
Attention
mla
Position encoding
rope-yarn
Pretraining tokens
6.0T
Post-training
sft, rlhf
OSI-approved
no
Data released
no
Training code
not released

Benchmarks

Each score carries the date it was published; we never infer or interpolate missing scores.

Code

HumanEval 90.2 as of 2024-06-17 source ↗
LiveCodeBench 43.4 as of 2024-06-17 source ↗

Recommended use cases

  • repo-scale code generation
  • multi-language code assistance
  • code reasoning at MoE economics

Available quantizations

GGUF llama.cpp's container; the common local format, k-quants from Q2 to Q8. runs on llama.cpp, Ollama
MLX Apple MLX 4/8-bit layout for Apple silicon. runs on Apple MLX

Verified via the Hugging Face model tree ↗. Community quantizations change over time; the families shown are those with published weights at audit time.

Notable innovations

  • · MoE code model at GPT-4 Turbo HumanEval parity
  • · 338 programming languages
  • · Context extended to 128K for repo-scale tasks

Lineage

Coder branch of the V2 line, derived from DeepSeek-V2 base (not from the V2-Chat sibling); later folded into V2.5.

Sources