Local LLMs in 2026 — the complete benchmark report on portable hardware

TL;DR

If you have €100, install Phi-3 mini 3.8B Q4 on a Raspberry Pi 5 and accept the limits. If you have €450, install Mistral 7B Q4 on a generic Intel mini PC and you have a credibly capable local-LLM agent. If you have €1,099, install Llama 3 8B Q4 on a Mac Mini M4 24 GB and you're competitive with small cloud API calls. If you have €1,899, install Llama 3.3 70B Q4 on a Mac Mini M4 Pro 48 GB and you're within striking distance of Claude Sonnet on most agent workloads. Anything beyond is a Mac Studio M3 Ultra or a custom workstation, with diminishing returns for personal use.

The numbers are below. The interpretation is the body of this guide.

1. Methodology — what we measured

Every hardware-model combination went through the same five agentic tasks, in the same order, against the same agent runtime (Hermes Agent 2026.4.4 wherever supported, ZeroClaw for cloud-free Mac Mini setups, Nanobot for the Pi 5 ceiling).

Tasks:

1. Single-step tool call — read a file, summarise. Baseline. 2. Multi-step planning — three or more tool calls in sequence. 3. Browser automation — navigate to a URL, extract structured data. 4. Long context — ingest a 50K-token document, answer four follow-ups referencing different sections. 5. Error recovery — request a tool that doesn't exist, score the recovery behaviour.

Metrics per task: - Pass rate (correct end-to-end completion, %) - Tokens per second average (end-to-end, including LLM inference and tool round-trips) - Power draw average (sampled at 1 Hz across the run)

Hardware tiers: - Raspberry Pi 5 (8 GB) - Lenovo ThinkCentre M75q (used, 16 GB Ryzen 5) - Geekom IT13 (32 GB i7) - Mac Mini M4 (16 GB) - Mac Mini M4 (24 GB) - Mac Mini M4 Pro (48 GB) - Mac Studio M3 Ultra (192 GB) - Custom workstation with RTX 5080

Models: - Phi-3 mini 3.8B Q4_K_M - Mistral 7B Q4_K_M - Llama 3 8B Q4_K_M - Llama 3.3 70B Q4_K_M - Qwen 2.5 72B Q4_K_M

Not every combination makes sense — we only ran combinations where the hardware can plausibly load the model.

2. Headline benchmark table

Reading the table: pass rate is the share of our 5-task suite the agent completed correctly without manual intervention. Cost @ 24/7 is just electricity at €0.30/kWh — not amortised hardware.

3. What the data actually says

Three patterns dominate.

3.1 Apple Silicon dominates per-watt and per-dollar at small form factor

The Mac Mini M4 24 GB at €1,099 hits 71% pass rate on Llama 3 8B Q4. The custom RTX 5080 at €1,800+ hardware cost hits 73% on the same model. The Mac Mini draws 18W typical; the RTX 5080 draws 195W. For agent workloads where the LLM call latency is one component among many (network round trips, tool execution, etc.), Apple Silicon's lower absolute throughput is offset by its meaningful power advantage and zero discrete-GPU ceremony.

If you're optimising for tokens-per-second-per-dollar on a single small machine, Apple wins.

If you're optimising for raw throughput at any cost, dedicated NVIDIA GPUs still dominate — but you're not in the “pocket AI” universe at that point.

3.2 70B-class is the frontier of useful local on €1,899 hardware

The Mac Mini M4 Pro 48 GB at €1,899 runs Llama 3.3 70B Q4 at 9.4 tok/s with an 84% pass rate. That last number is the relevant one. 84% means roughly 4 out of 5 of our standard agentic tasks complete without manual intervention. The remaining 16% — typically multi-step planning where the model gets confused — would have benefited from cloud-tier reasoning, but they fail in recoverable ways.

For most personal and small-team workloads, that pass rate is the threshold of usability. Below 80%, you're constantly intervening. Above 80%, the agent feels reliable enough to trust with multi-step delegations.

The €1,899 hardware investment, amortised over 24 months, is €79/month + €9/month electricity = €88/month total. A heavy Claude Sonnet user can spend €100-200/month on API calls alone. The break-even is real for sustained workloads.

3.3 The Pi 5 is a real device, not a toy — but you have to accept the limits

The Pi 5 + Phi-3 mini 3.8B at Q4 hits 35% pass rate. That's not a “just don't” number — for narrow tool-use workflows (filesystem reads, structured queries, rate-limit-fallback to a cloud LLM), 35% means the agent completes a third of tasks without bothering the cloud. That's genuinely useful. The €2.10/month electricity bill adds context: you can leave a Pi 5 running this for years and barely notice.

What it cannot do: complex multi-step reasoning, browser automation beyond minimal tasks, anything requiring 7B+ model capability. We benchmarked Mistral 7B on the Pi 5 — it loads, it runs at 1.6 tok/s, it fails 80% of our test tasks on timeout. Don't try.

4. Per-model recommendations

4.1 Phi-3 mini 3.8B

The Pi 5 model. €2/month all-in once the hardware is paid for. Useful for narrow workflows where the agent calls a cloud LLM as fallback for hard tasks. Not a primary LLM for general agent work.

4.2 Mistral 7B

The mini PC sweet spot. Runs comfortably on €450-700 mini PCs. 60-70% pass rate is realistic. Apache 2.0 license means you can ship products on top of it without negotiation.

4.3 Llama 3 8B

The Mac Mini M4 24 GB sweet spot. 71% pass rate at 38 tok/s. Strong multilingual performance. License is non-commercial-restricted but permits broad use.

4.4 Mistral Small 22B

The mid-size model that became serious in early 2026. 76% pass rate on a 24 GB Mac Mini M4 at 21.6 tok/s. Apache 2.0. We expect this size class to be the dominant 2026 deployment target.

4.5 Llama 3.3 70B

The 70B-class endgame at small form factor. 84% pass rate on a 48 GB Mac Mini M4 Pro at 9.4 tok/s. The combination most likely to make you stop calling cloud APIs.

4.6 Qwen 2.5 72B

Slightly outperforms Llama 3.3 70B on coding-specific tasks (86% vs 84% on our suite, with the gap widening on code-heavy subtasks). Apache 2.0. Worth considering if your agent does meaningful code generation.

5. Per-hardware recommendations

Pi 5: Phi-3 mini 3.8B Q4. Don't try larger. Use a cloud-LLM fallback for complex tasks.

ThinkCentre M75q (used): Mistral 7B Q4. Watch out for thermal throttling on sustained loads — older fan, possibly worn paste.

Geekom IT13 (32 GB): Mistral 7B Q4 for primary, Phi-3 mini 3.8B Q4 as a quick-fallback when the main model is loaded but you need a fast response.

Mac Mini M4 (24 GB): Llama 3 8B Q4 as primary. Mistral Small 22B Q4 for harder tasks. Both fit comfortably with agent overhead.

Mac Mini M4 Pro (48 GB): Llama 3.3 70B Q4 as primary. Don't bother with smaller models on this hardware — the headroom is wasted.

Mac Studio M3 Ultra (192 GB): Llama 3.3 70B Q4 at 22 tok/s, Qwen 2.5 72B Q4 at 19 tok/s. The throughput tier that makes 70B-class models actually pleasant to use.

Custom workstation with RTX 5080: Llama 3 8B Q4 at 76 tok/s for small-model speed, Llama 3.3 70B Q4 at 38 tok/s for capability. If you have it, you don't need our advice.

6. The cloud-vs-local cost crossover

Cloud LLM costs scale with usage. Local LLM costs scale with hardware (amortised) and electricity (linear). The crossover depends on your actual usage pattern.

For light usage (≤500K input tokens/month), cloud is cheaper. The hardware never pays back.

For moderate usage (500K-5M tokens/month), the crossover happens around 12-18 months on a Mac Mini M4. After that point, every additional cloud API call is dead weight.

For heavy usage (5M+ tokens/month, which a sustained agent workload easily reaches), local LLMs are dramatically cheaper after 6-9 months on appropriate hardware.

Concrete example. An always-on agent making 3 cloud API calls per minute averages 4M-8M Claude Sonnet input tokens per month, depending on context size. At Anthropic list prices that's €120-240/month. A €1,099 Mac Mini M4 24 GB with Llama 3 8B running locally amortises to €46/month + €5 electricity = €51/month. Break-even at month 5-7.

7. What we did NOT measure

Three things deliberately out of scope:

• Long-term reliability. Two weeks of testing cannot tell you what
• Adversarial reasoning quality. Our pass-rate measure ignores cases
• Tooling ecosystem fit. Some agents work better with some local

8. Buying advice, condensed

If you're planning a local-LLM purchase in mid-2026:

1. Don't over-buy. A €1,099 Mac Mini M4 24 GB does 71% pass rate at ~€50/month all-in. That's better than most people need. 2. Don't under-buy. A €450 mini PC running Mistral 7B at 62% pass rate is fine for tinkering. It's not fine for serious daily use. 3. Avoid the Pi 5 trap. The Pi 5 is real but limited. If your workload doesn't fit Phi-3 mini 3.8B's capability, the Pi 5 is the wrong device. Cheaper to skip the experiment and go straight to a €450 mini PC. 4. 48 GB unified memory is the breakpoint. Below it, you're in small-model territory. At 48 GB, 70B-class models become tractable. This is the threshold worth saving up for. 5. Cloud LLM API costs accumulate quietly. Run the actual numbers for your workload before claiming local AI “saves money.”

9. What we expect from late 2026 / 2027

• Llama 4 (rumoured Q3 2026) probably brings frontier-tier
• Apple's M5 likely pushes unified memory to ≥256 GB on Mac Studio
• Sub-7B models continue to surprise. The next 18 months of
• Quantisation techniques keep improving. Q4 today is good; Q3 with

We re-run this benchmark quarterly. The next update is scheduled for late July 2026. Subscribe to [the newsletter](/newsletter) to get it in your inbox.

Related guides

• [Pocket AI complete guide](/guides/pocket-ai-complete-guide)
• [Edge AI hardware buyer's guide 2026](/guides/edge-ai-hardware-2026)
• [Self-hosted AI landscape report](/guides/self-hosted-ai-landscape-2026)
• [Methodology — how we test](/methodology)