About the figures in this article. It was first published as a 21-day benchmark of four machines, with tables of tokens per second, watts and cost per million tokens. We have no raw measurements we can publish to back those tables, so they have been removed. What remains is the reasoning, with a few orders of magnitude marked as estimates. For a decision that matters, measure on the hardware you are considering, with your own prompts. Note added in October 2026.
There is a default assumption running through most self-hosted AI tutorials in 2026: get a GPU. The assumption is so common that the alternative rarely gets a serious look. It deserves one: for many self-hosted workloads the default is wrong.
This is the article we wished existed when we were buying hardware.
The four hosts
| Host | Spec | Idle power, published | Price (estimate, mid-2026 EU) |
|---|---|---|---|
| Raspberry Pi 5 8 GB | ARM Cortex-A76, no GPU | about 4 W (bare board, typical, per Raspberry Pi) | about €140 with case and PSU |
| Intel NUC 13 i7 | i7-1360P, Iris Xe iGPU | no maker figure found | about €620 |
| Mac Mini M4 16 GB | Apple M4, 10-core, unified memory | 4 W (Apple) | about €750 |
| GPU box | Ryzen 5 7600 + RTX 4060 Ti 16 GB | none: depends on the parts | about €1,200 self-built |
Sources: Raspberry Pi's power supply documentation and Apple's Mac mini power consumption page, read on 17 October 2026. For the other two, measure at the wall with a plug-in meter.
The comparison assumes the same runtime everywhere (Ollama) and the same quantised models.
The five models
tinyllama:1.1b-chat-q4_0(~640 MB resident)qwen2.5-coder:3b-instruct-q4_K_M(~2.0 GB)qwen2.5-coder:7b-instruct-q4_K_M(~4.5 GB)qwen2.5:14b-instruct-q4_K_M(~8.5 GB)qwen2.5:32b-instruct-q4_K_M(~19 GB)
The 32B model is about 19 GB. It does not fit in 16 GB of memory, on the Mac Mini or on the NUC, without swapping, which makes it impractical there. On the GPU box it has to be split between the 16 GB of the card and system memory.
The three workloads
- Single-prompt classification: 200 input tokens, ~5 output. The inbox-classification shape.
- Code review on a 220-line diff: 3,500 input tokens, ~600 output. The PR-review shape.
- Long-context summarisation: 12,000 input tokens, ~400 output. The doc-summarisation shape.
What to expect, workload by workload
| Workload | Pi 5 | NUC 13 | Mac Mini M4 | GPU box |
|---|---|---|---|---|
| Published generation speed | about 11 tokens per second on 1B models, about 5 on 3B (Slips) | none for this chip; nearest Vulkan entries: Intel Core 1100 series 10.39, Core Ultra 100 series 8.21 tokens per second on a 7B model | 24.11 tokens per second on a 7B model (llama.cpp, Metal) | none for this card; nearest Vulkan entries: RTX 4060 laptop 59.53, RTX 4070 92.29 tokens per second on a 7B model |
| Classification, 1B model | fine | fine | fine | fine, and wasteful |
| Code review, 3B model | slow: minutes per review | usable | interactive | fastest |
| Code review, 7B model | too slow for interactive use | slow | usable | fastest |
| 12 K of context, 7B model | not practical | works, slowly | works | fastest |
The verdicts are our reading of the first row, not measurements of ours. For the Pi 5 the figure is a published measurement on that board: the Slips project's tests give about 11 tokens per second on 1B models and about 5 on 3B models, with Ollama at Q4_K_M. The Mac Mini M4 figure is llama.cpp's Apple Silicon scoreboard (Llama 2 7B at Q4_0, 10-core GPU). For the NUC and the GPU box, the Vulkan scoreboard has no entry for these exact parts, so the figures are its nearest chips, same model and test. Both scoreboards read on 17 October 2026.
Single-prompt classification (1B model)
Every host handles this. A 1B model answers a 200-token prompt with a 5-token label quickly enough on any of them that, at a hundred or so messages a day, the difference between hosts is invisible to the user. The GPU is the fastest in absolute terms and the most wasteful: the whole machine is powered to run a model a Pi can run.
Code review on 3.5 K input + 600 output (3B model)
This is the workload where the GPU starts to matter. A review that comes back in a few seconds feels live; one that takes minutes does not. The Pi 5 is technically capable but uncomfortable for interactive use at this size.
The Mac Mini is the interesting case: fast enough for interactive use, at a fraction of the GPU box's power draw. For a team that reviews ~10-20 PRs a day, it is the cost-effective answer.
Code review on 7B
7B on the Pi is unusable for interactive work. The NUC's iGPU helps a bit, but a review still takes long enough to break the flow. The Mac Mini is the cheapest host on which it stays usable. The GPU box is the only one that feels fast, and it pays for that with the highest power draw under load.
Long-context summarisation (12 K in, 400 out, 7B model)
The Pi 5 is the wrong tool here: with a 7B model and 12 K tokens of context, memory is nearly full and reading the prompt alone takes far too long. The NUC gets through it slowly. The Mini handles it more comfortably thanks to unified memory. The GPU is fast, but it is a whole rig for a workload that may run once a day.
Cost per month, and why cost per million tokens misleads
Two costs: the hardware, and the electricity to keep it on. With electricity at €0.21/kWh and the hardware amortised over 36 months, from the prices in the first table, the published idle figures, and an assumed idle draw where no maker gives one:
| Host | Hardware per month | Electricity per month, idle |
|---|---|---|
| Pi 5 | about €4 | under €1 |
| NUC 13 | about €17 | about €1.50 if it idles at 10 W |
| Mac Mini M4 | about €21 | under €1 |
| GPU box | about €33 | about €6 if it idles at 40 W |
Under load the electricity line grows, most of all on the GPU box.
A cost per million tokens can be computed from figures like these, but only for a machine that generates around the clock, which is not how personal or small-team hosts are used. The GPU is cheapest per token only if you keep it busy. For intermittent traffic: i.e. most personal and small-team use: its idle power and its purchase price are paid whether or not it works.
The Mac Mini is the economical choice for moderate steady traffic. The Pi has the lowest monthly bill if your traffic is small.
When each host wins
The recommendation:
- Raspberry Pi 5: edge classification, agent orchestration, small models (≤3B). Wrong choice for anything 7B+. Right choice when you want a sub-€10/month total cost of ownership.
- Intel NUC 13 (or Mini PC equivalent): bridge tier. Better than the Pi at 7B, less efficient than the Mac Mini. Right choice when you can't run macOS for whatever reason and need 7B capability locally.
- Mac Mini M4: our pick as the all-rounder for self-hosted AI in 2026. Cheaper to own and to keep on than the GPU box, and efficient under load. Loses on raw throughput when traffic is high. Right choice for most individuals and small teams.
- Single-GPU box: high steady throughput, bigger models, batch workloads. Right choice for small companies serving multiple users concurrently. Wrong choice if the box sits idle 90% of the day: the Mac Mini is then the more economical machine.
The shape of your traffic matters more than the headline benchmark
Three lessons:
- Idle power is a hidden cost. A box drawing 40 W idle costs about €74 a year in electricity at €0.21/kWh before doing any work. A machine that idles under 10 W has a structural advantage on intermittent workloads.
- Tokens-per-second is not the only axis. Energy per token and hardware cost per month matter for self-hosting in ways they do not for hyperscale.
- The Apple Silicon advantage is real. Unified memory, low idle power, and decent throughput at small-to-medium model sizes make the Mac Mini the default answer for most of the cases above.
Reference reading
- Pocket AI hardware buyer's guide 2026
- Raspberry Pi 5 as an AI agent host
- Mac Mini M4 spec sheet
- Intel NUC 13 spec sheet
- Self-hosted AI cost calculator
The right answer is “it depends on your traffic shape and whether you care about Wh as much as you care about ms.” Most self-hosted AI users care about Wh more than they realise. The Mac Mini is, in 2026, the boring answer that is correct most of the time.