Pocket AI: which self-hosted agents run on small hardware

TL;DR

Pocket AI means running self-hosted AI agents, and optionally local language models, on small hardware you own. This guide answers one question: which agent runs on which class of device. It covers six device classes (single-board computers, mini PCs, Apple Silicon, Framework laptops, refurbished phones, edge endpoints), matches them with the self-hosted agents we track (OpenClaw, Hermes, Nanobot, NanoClaw, IronClaw, ZeroClaw, Moltworker), and gives six agent-first setups.

Three other pages carry the rest, so this one does not repeat them:

• What to buy, by budget: the hardware buyer's guide.
• How fast a local model runs on each machine: Local LLM benchmarks, published measurements with their setup.
• One device in detail: its spec sheet under /pocket.

The short version: an agent that calls a cloud model (Claude, GPT, OpenRouter) runs on a Raspberry Pi 5; an agent that drives a browser wants a 32 GB mini PC; an agent that runs its own model locally wants Apple Silicon with as much memory as you can afford.

Section 1: why pocket AI

The cloud-AI bet, “datacenters with billions in capex are the only viable host for useful intelligence”, is the dominant 2025/2026 narrative. It is also incomplete.

Three things changed between late 2024 and 2026.

Hardware caught up. Raspberry Pi gives the Pi 5 “2–3× the speed of the previous generation”, and sells it with up to 16 GB of memory (product page). Apple's Mac mini with the M4 Pro chip can be ordered with 64 GB of unified memory (Mac mini (2024) tech specs), enough to load Llama 3.3 70B, a 43 GB file at 4-bit in the Ollama library.

Self-hosted agents got small enough for small machines. Nanobot ships Linux ARM64 wheels, so a 64-bit Raspberry Pi takes a plain pip install (README, read on 18 October 2026; until then this line said Nanobot fits in a few thousand lines of Python, which was its February 2026 launch claim, not its size today). ZeroClaw is a single Rust binary that runs on any Raspberry Pi, the Zero 2 W included (Raspberry Pi setup). Hermes Agent and OpenClaw run on an ordinary Linux box, in a container if you set one up: neither sandboxes by default. None of them needs the “16 vCPUs to run an agent” assumptions of 2024.

Local models got fast enough to matter. On llama.cpp's published scoreboards, the same 7B model at 4-bit writes 24.11 tokens per second on an Apple M4 and 19.91 on AMD Ryzen 7000-series integrated graphics. Cloud models still lead on capability, but “run locally” is now a real option for the narrower parts of an agent's work.

The pocket AI bet, restated: for many personal and small-team self-hosted AI use cases, a €100-€2,000 host running an appropriate agent and (optionally) a local model is a credible alternative to cloud-AI services, with no data residency questions.

Section 2: the six device classes

Every small host for self-hosted AI fits into one of six classes. What decides which agent fits is memory, the CPU architecture (ARM or x86), the operating system, and whether the device is meant to stay on.

2.1 Single-board computers (SBCs)

The Raspberry Pi 5 is the canonical example; the Orange Pi 5 Plus, the Khadas Edge2 and the Radxa Rock 5B+ are the alternatives. All are ARM-based and run Linux as a first-class OS.

Strengths: cheap, small, quiet, and low power: Raspberry Pi gives a typical bare-board current of 800 mA for the Pi 5, about 4 W, before storage and load (power supply documentation). The Pi 5 has the biggest software ecosystem of any SBC: Hermes Agent, Nanobot, ZeroClaw and IronClaw all publish Linux ARM64 builds.

Limits: 16 GB of memory at most on the Pi 5 and the Orange Pi 5 Plus, per their makers' pages, which is tight for browser-automation tools. For local models, the published Pi 5 figures stop at 3B models, at about 5 tokens per second (Slips project).

2.2 Mini PCs

The Intel NUC line (now made by ASUS) and its generic competitors from Beelink, Geekom and Minisforum: small x86 boxes, usually with 16 to 64 GB of RAM.

Strengths: x86 means anything that runs on Linux runs here, including the heaviest agent runtimes and Chromium-based browser automation. A 7B or 8B model at 4-bit (a 4 to 5 GB download) fits next to the agent in 32 GB.

Limits: mains power and a desk. Local-model speed depends on the integrated graphics: on the llama.cpp Vulkan scoreboard, the same 7B model runs from 2.93 tokens per second on an Intel N150 to 19.91 on AMD Ryzen 7000-series graphics.

2.3 Apple Silicon (Mac mini, MacBook, Mac Studio)

Unified memory is the reason local-model users pick a Mac: the GPU reads the model from the same fast memory as the CPU. The Mac mini M4 goes to 32 GB at 120 GB/s, the M4 Pro to 64 GB at 273 GB/s, and the Mac Studio M3 Ultra to 256 GB at 819 GB/s (Apple's tech specs for the Mac mini (2024) and the Mac Studio (2025)).

Power: Apple gives 4 W idle and 65 W maximum for the Mac mini M4, 5 W and 140 W for the M4 Pro (Apple).

Limits: macOS. Asahi Linux exists but is not production-grade for server workloads, so Linux-only agents and tools are a poor fit. Memory is chosen at purchase and cannot be added later.

2.4 Framework laptops

Framework Laptop 13 and 16 are the credible “laptop that doubles as an agent host” option: repairable, modular, with strong Linux support and up to 64 GB of RAM.

Strengths: one machine for both agent hosting and daily work, and a mainboard that can later become a desktop.

Limits: laptops as servers always involve compromises: battery hardware that ages, sleep behaviour that's never quite right for always-on workloads, fan curves tuned for portability rather than sustained load.

2.5 Refurbished and repurposed phones

The PocketClaw origin story. A used Android phone running Termux and proot can host a very limited agent that calls a cloud model.

Strengths: cheap, low-power, and good for learning.

Limits: 1 to 8 GB of RAM depending on the phone, a software stack that is a constant fight (Termux is not standard Linux, proot adds overhead), no production deployment.

2.6 Edge endpoints (Pi Zero 2 W and similar)

Boards too small to host a primary agent (the Pi Zero 2 W has 512 MB of RAM), but useful as tool hosts in a multi-device setup: read sensors, control GPIO, run small utilities.

Pair them with a primary host (Pi 5, mini PC, Mac mini) and the Zeros act as the eyes, ears and hands of a single agent. No local model of any kind runs on them.

Section 3: self-hosted agents on pocket hardware

The agent-by-device matrix: our reading of each agent's requirements against each device class, not a test of every pair.

Reading the table: “yes” means the agent's requirements fit the device class; a qualifier names the constraint; “no” means don't try. NanoClaw needs Docker on every host: its README lists macOS, Linux and Windows through WSL2, and until October 2026 this row said macOS only. IronClaw publishes builds for macOS, Linux (x86_64 and ARM64) and Windows; until then its row said light use only. ZeroClaw's prebuilt binaries cover every Raspberry Pi and, experimentally, Android through Termux (Android setup); until October 2026 its row assumed it only runs local models. Moltworker runs OpenClaw in a Cloudflare Sandbox container behind a Worker, not on your hardware.

The realistic match-ups for first deployments:

• Raspberry Pi 5 (8 GB) + Hermes Agent without the browser tool: the agent calls Claude or GPT for reasoning and uses non-browser tools (filesystem, shell, MCP servers). The cheapest credible setup.
• 32 GB mini PC + Hermes Agent with every tool: browser automation works, and a 7B local model fits as a fallback.
• Mac mini M4 (24 GB or more) + ZeroClaw + an 8B model: fully local, no cloud calls.
• Mac mini M4 Pro (48 or 64 GB) + ZeroClaw + Llama 3.3 70B: the largest model class that loads on a small machine; 64 GB leaves room for the context and the agent.
• Old Android phone + Termux + Nanobot: a hobbyist proof of concept. Don't run anything that matters here.

Section 4: local LLM speeds, in brief

An earlier version of this guide had a table of tokens per second, “pass rates” and monthly running costs for each device class. Nothing was measured to produce it, so it is gone. These are published measurements instead, each with its setup:

Sources, read on 17 October 2026: the Slips project's Raspberry Pi 5 tests, llama.cpp's Vulkan scoreboard and Apple Silicon scoreboard. The builds, the prompt-reading speeds and more machines are in Local LLM benchmarks.

Two things these numbers do not say. They measure speed, not how good the answers are, so they cannot tell you whether a local model can do an agent's job as well as a cloud one. And neither llama.cpp scoreboard has a 70B row, so there is no published speed here for the largest models a Mac mini M4 Pro can load.

Section 5: six setups, agent first

The same six questions come up again and again. Each gets one agent, one class of host and one model plan. Prices, exact models and the trade-offs between machines are in the buyer's guide.

5.1 Start cheap

Raspberry Pi OS, Docker, Hermes Agent with the browser tool disabled, Tailscale to reach it from your laptop. The agent calls Claude or OpenRouter; the Pi runs the tools. If something breaks, someone in the largest hobbyist community in computing has documented the fix.

5.2 Real work, browser included

Debian or Ubuntu LTS, Docker, Caddy as reverse proxy, Tailscale, Hermes Agent with the full tool set. Prefer a cloud model for the main work and keep a local 7B model (Mistral 7B is a 4.4 GB download at Q4_K_M) for cheap subtasks: it removes rate-limit anxiety.

5.3 A local model as primary

macOS, Ollama with an 8B model, and an agent that is happy with local models: NanoClaw (through its Ollama provider skill) or ZeroClaw (Ollama is one of its providers). 24 GB leaves room for the model, the agent and the system.

5.4 70B-class local models

Llama 3.3 70B at Q4 is a 43 GB file; Qwen 2.5 72B is 47 GB. At 48 GB of unified memory the 70B model loads with little left over; 64 GB is the configuration that leaves room for the context and the agent. This is the largest model class that fits on a small machine.

5.5 One machine for work and AI

Linux on a Framework Laptop 13, Hermes Agent in Docker, Ollama with a 7B model for personal use. Power management is the main concern: configure the agent to pause on battery and run when plugged in.

5.6 Distributed tools

The primary runs Hermes Agent and an MCP server registry. Each Pi Zero runs a tiny MCP server exposing one tool (sensor reader, GPIO controller, file watcher). Each endpoint is single-purpose and replaceable, and the MCP protocol keeps the integration clean.

Section 6: what pocket AI is not

Three things pocket AI is not, and we want to be honest about it.

It is not a frontier-AI replacement. The largest cloud models still beat any local model on hard tasks: multi-step reasoning across long contexts, code generation at scale, novel problem-solving. If your agent needs frontier intelligence, no pocket hardware substitutes for a cloud API.

It is not always cheaper. A €450 mini PC paid back over 24 months is about €19 a month, plus electricity. A heavy Claude API user can spend that in two days; light users spend nothing close. Run the numbers for your own workload before claiming “saves money”.

It is not zero-maintenance. Self-hosting means OS updates, agent updates, certificate renewals, and debugging when something breaks. If you want zero ops, hosted services exist for a reason.

Section 7: what to watch

What moved since this guide was first written in April 2026, and what we are watching:

• Hardware. Apple's spec pages, read on 16 October 2026, list the Mac mini with M6 and M5 Pro chips and the Mac Studio with M5 Max and M5 Ultra chips; the hardware buyer's guide covers what that changes. The Raspberry Pi 5 has been sold with 16 GB since January 2025.
• Local models. Small models keep improving; the question for an agent is whether one is good enough for a given narrow task, which no speed figure answers.
• Agents. Our base case is that the field consolidates around a few defaults, with NanoClaw for container isolation by default, ZeroClaw for small hosts and IronClaw for tools sandboxed in WebAssembly. Projects without a specific niche will struggle.

Closing notes

Pocket AI is the bet that owning the layer of intelligence that runs your own data is worth the operational work it costs. It holds for users with the right workloads, the right expectations, and the willingness to do the ops.

Pick the agent first, then the smallest device class that runs it:

1. An agent that calls a cloud model: Raspberry Pi 5 (8 GB).
2. An agent that drives a browser: a 32 GB mini PC.
3. An agent that runs its own model: a Mac mini M4 (24 GB) or M4 Pro (48 or 64 GB).

Device spec sheets live at /pocket, the agents at /agents, the CVE tracker at /cves, and the newsletter at /newsletter.

Related guides

• Hardware buyer's guide 2026: what to buy, by budget
• Local LLM benchmarks: published speeds, model by machine
• The complete OpenClaw timeline
• Self-hosted AI agents 2026: landscape report
• 5 best OpenClaw alternatives
• Migrate from OpenClaw to Hermes Agent