Specs
- Memory
- 32 GB LPDDR5X
- Bandwidth
- 136 GB/s
- NPU
- 48 INT8 TOPS
- Power
- 37 W
- Form factor
- soc
- Interconnect
- none
- Released
- 2024-09
What it runs (single unit, Q4_K_M, 4K context)
| Model | Fits? | Decode ceiling |
|---|---|---|
| Llama 3.1 8B Instruct | yes | ~27 tok/s |
| Llama 3.3 70B Instruct | no | — |
| Qwen 2.5 72B Instruct | no | — |
| DeepSeek-V3 | no | — |
Ceiling is the theoretical rooflineruntimeA performance model that bounds throughput by either compute or memory bandwidth, whichever is the limiting resource for an operation's arithmetic intensity. Open full entry ; open the explorer to set quant, context, and runtime and see the realistic range.