Llama 3.2 3B
225 tokens a second on Windows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM. About $1.14 per million tokens here, against OpenRouter's $0.33: the API is 3.5× cheaper. Run it here for privacy or to keep work on your own machines, not to save money.
13 builds of Llama 3.2 3B, 14 signed runs on real machines: different packagers, precisions and engines, each a different file with its own numbers. 3.2B parameters, as the vendor states.
trained 128K · proved 128K context, by bartowski Q4_K_M
225.2 tok/s peak, by unsloth IQ1_S
Speed against price, every machine
Up and left is faster and cheaper · the line joins machines no other beats on both · price on a log scale · dashed: OpenRouter
What it fits on
The smallest build needs about 1.6 GB; 13 of 13 kinds of machine on the network can hold it. Bars: each kind's memory (its card's, else RAM) · ticks: every build, at the size it held plus a fifth for the prompt.
- NVIDIA GeForce RTX 3050 Laptop GPU4 GB on the card1 of 7
- NVIDIA GeForce GTX 16504 GB on the card1 of 7
- AMD Radeon(TM) Vega 8 Mobile Graphics7 GB RAM1 of 7
- Windows, CPU only · 12 CPUs7 GB RAM1 of 7
- apple8 GB RAM1 of 7
- Apple M18 GB RAM1 of 7
- NVIDIA GeForce RTX 3080 Ti12 GB on the card2 of 7
- Tesla T415 GB on the card2 of 7
- Intel(R) UHD Graphics 73016 GB RAM2 of 7
- Intel(R) Iris(R) Xe Graphics16 GB RAM2 of 7
- Apple M416 GB RAM2 of 7
- 2× Tesla T429 GB on the cardevery build
- Linux, CPU only · 4 CPUs31 GB RAMevery build
How far back it remembers, machine by machine
Filled: the needle came back at that prompt size · outlined: asked and lost · blank: not asked · trained for 128K. 1 walk ended because the model forgot, 0 because the machine could not hold the prompt.
| machine · build | 4K | 8K | 16K | 32K | 64K | 128K | why it stopped |
|---|---|---|---|---|---|---|---|
| Gram ·a5be46Tesla T4 · Q4_K_M by bartowski | 128K recalled at every size asked | ||||||
| Gram ·a5be46Tesla T4 · Q3_K_L by bartowski | 128K recalled at every size asked | ||||||
| Gram ·a5be46Tesla T4 · IQ3_M by bartowski | 128K recalled at every size asked | ||||||
| Gram ·a5be46Tesla T4 · Q8_0 by bartowski | 128K recalled at every size asked | ||||||
| Gram ·a5be46Tesla T4 · F16 by bartowski | 128K recalled at every size asked | ||||||
| Kunal HindochaNVIDIA GeForce RTX 3080 Ti · F16 by unsloth | 16K | ||||||
| Kunal HindochaNVIDIA GeForce RTX 3080 Ti · IQ1_S by unsloth | — | ||||||
| Kunal HindochaNVIDIA GeForce RTX 3080 Ti · Q4_0 by unsloth | — | ||||||
| Kunal HindochaNVIDIA GeForce RTX 3080 Ti · Q4_K_M by unsloth | — | ||||||
| Kunal HindochaNVIDIA GeForce RTX 3080 Ti · IQ3_XXS by unsloth | — | ||||||
| Kunal HindochaNVIDIA GeForce RTX 3080 Ti · IQ2_XXS by unsloth | — | ||||||
| Gram ·a5be46Tesla T4 · IQ1_S by unsloth | — the model lost the needle at 4K | ||||||
| Kunal HindochaNVIDIA GeForce RTX 3080 Ti · Q8_0 by unsloth | — | ||||||
| nitin singhApple M4 · IQ1_S by maziyarpanahi | — |
Carbon per million tokens
Lowest build on each machine · 1 of 1 read off a power sensor, the rest modelled from the parts · fewer grams is better
- Gram ·a5be46983 g7.49 J/tok · sensor
Carbon is location-based: signed energy × the yearly average of the grid where the machine signed its place (world grid average (place not signed); Ember Global Electricity Review 2025 (2024 generation), location-based). Facility overhead is not included. A machine on its own solar is not credited until it can prove it.
Measured by
The people whose machines signed these numbers
proved by at least one build: Tool calling · Tool loop finished · Faithful to tool results · Structured output · Sustained output · Writes code · Reasons about code · Synthesises sources · Follows instructions
Every build
Each its own file with its own page: the machines that ran it and their signatures.
| build | engine | whose work | memory | context proved | best tok/s | machines |
|---|---|---|---|---|---|---|
| Q4_K_M by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M | ollama | quantizer · bartowski | 24.9 GB | 128K | 86.9 | 1 |
| Q3_K_L by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q3_K_L | ollama | quantizer · bartowski | 24.7 GB | 128K | 78.8 | 1 |
| IQ3_M by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:IQ3_M | ollama | quantizer · bartowski | 24.5 GB | 128K | 72.0 | 1 |
| Q8_0 by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0 | ollama | quantizer · bartowski | 26.5 GB | 128K | 62.8 | 1 |
| F16 by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:F16 | ollama | quantizer · bartowski | 26.9 GB | 128K | 38.3 | 1 |
| F16 by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:F16 | ollama | quantizer · unsloth | 9.9 GB | 16K | 94.9 | 1 |
| IQ1_S by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:IQ1_S | ollama | quantizer · unsloth | 1.6 GB | — | 225.2 | 2 |
| Q4_0 by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:Q4_0 | ollama | quantizer · unsloth | — | — | 205.1 | 1 |
| Q4_K_M by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:Q4_K_M | ollama | quantizer · unsloth | — | — | 203.9 | 1 |
| IQ3_XXS by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:IQ3_XXS | ollama | quantizer · unsloth | — | — | 199.5 | 1 |
| IQ2_XXS by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:IQ2_XXS | ollama | quantizer · unsloth | — | — | 124.2 | 1 |
| Q8_0 by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:Q8_0 | ollama | quantizer · unsloth | — | — | 72.8 | 1 |
| IQ1_S by maziyarpanahihf.co/MaziyarPanahi/Llama-3.2-3B-Instruct-GGUF:IQ1_S | ollama | maziyarpanahi | — | — | 45.7 | 1 |
Fine-tunes, merges and abliterations of this member, each its own model with its own page. None of their numbers are counted above.