< Back to all hacks

#20 V8 Heap 192 MB Minimum

RAM
Problem
128 MB heap OOMs during startup compilation phase (~186 MB needed for ESM module linking).
Solution
192 MB as minimum viable heap. Survives boot peak, GC keeps it stable after.
Lesson
V8 heap peak at startup is transient. Steady-state usage is much lower, but the process must survive the spike.

Context

When Node.js boots an application that uses ESM (ECMAScript Modules), V8 must parse, compile, and link all modules before any application code executes. Unlike CommonJS, where modules are loaded on-demand during execution, ESM requires the entire module graph to be resolved up front. For OpenClaw, this means parsing hundreds of modules, their dependencies, and all transitive imports in a single synchronous phase.

On the Moto E2's 32-bit ARM V8, this compilation phase is the peak memory consumer. V8 allocates ASTs, bytecode, source maps, and module namespace objects all at once. The peak measured during startup was approximately 186 MB of V8 heap usage. After startup completes and the initial compilation objects are garbage collected, steady-state heap usage drops to 90-110 MB.

The problem: if --max-old-space-size is set below the startup peak, V8 attempts garbage collection, fails to free enough memory (because everything is still reachable during compilation), and terminates with a FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory.

Implementation

Set the heap cap in NODE_OPTIONS:

# In the gateway start script:
export NODE_OPTIONS='-r /root/hijack.js --expose-gc --max-old-space-size=192'

This was the initial working configuration discovered through binary search (Hack #16). The 192 MB cap provides ~6 MB of headroom above the 186 MB startup peak, which is enough for V8's GC to operate without triggering OOM.

The startup memory profile looks like this:

Time (s)    V8 Heap (MB)    Phase
0-2         20-40           Module resolution (file I/O)
2-8         40-120          Parsing + compilation
8-15        120-186         Module linking (PEAK)
15-20       186 -> 95       GC reclaims compilation artifacts
20+         90-110          Steady state (serving requests)

Verification

# Start the gateway and monitor heap during boot:
node --max-old-space-size=192 --expose-gc -e "
  const v8 = require('v8');
  setInterval(() => {
    const h = v8.getHeapStatistics();
    console.log('heap:', (h.used_heap_size / 1024 / 1024).toFixed(1), 'MB');
  }, 1000);
  require('./dist/cli.js');
"

# Verify the gateway survives startup:
curl -s http://localhost:9000/api/status
# Expected: JSON response after 15-20s boot

# Check peak RSS after startup settles:
ps -o pid,rss -p $(pgrep -f openclaw-gateway)
# Expected: RSS ~175 MB (V8 heap + native overhead)

Gotchas

  • 192 MB was the minimum for the proot-based stack. When the native node22-icu build (Hack #12) replaced proot, the startup peak shifted because proot's syscall translation overhead was eliminated. The minimum was progressively lowered: 192 -> 150 -> 112 MB
  • The startup peak is not deterministic. Module load order, file system cache state, and available system memory all affect it slightly. The 6 MB headroom at 192 MB was tight but sufficient across hundreds of boot cycles
  • --max-old-space-size only controls old space. V8 also uses new space (semi-space), code space, map space, and large object space. Total V8 memory is always higher than the old-space cap
  • On 1 GB RAM devices, the heap cap directly competes with Android's own memory needs. At 192 MB, the gateway's total RSS (~175 MB) plus Android's floor (~300 MB) leaves only ~500 MB for other processes
  • If startup OOMs, there is no partial boot. The gateway must be restarted from scratch. There is no way to resume from where it left off

Result

Heap CapBootSteady StateRSS
128 MBOOM at linking--
192 MBSurvives peak90-110 MB~175 MB
256 MBComfortable90-110 MB~195 MB
384 MBWasteful90-110 MB~260 MB

192 MB was the first production-viable heap cap, running stable for days under the proot stack. It was later superseded by 150 MB (Hack #21) and finally 112 MB on the native build. The key insight: the startup peak is the only constraint. Once past it, V8's GC keeps the heap well within bounds. Every MB saved on the heap cap is a MB returned to Android for system processes.