#16 V8 Heap Binary Search

Context

On the Moto E2 with 1 GB RAM, the V8 heap cap is the single most important tuning parameter. Set it too high and the gateway RSS bloats, pushing Android into low-memory killer territory. Set it too low and V8 OOMs during startup before the gateway can serve a single request.

The default V8 heap on 32-bit systems is around 700 MB, which is absurd for a device with 920 MB total physical RAM. But what is the actual minimum? There is no way to know without testing. V8's memory usage depends on the specific application: number of modules, code complexity, data structures, and the interplay between ESM compilation, JIT compilation, and garbage collection.

The approach: treat it like a binary search problem. Start with a known-good value (384 MB, confirmed working), a known-bad value (64 MB, obviously too small), and bisect until the minimum stable value is found.

Implementation

The binary search was conducted over multiple gateway boot cycles. Each test consisted of: set the heap cap, start the gateway, wait for full startup (HTTP response on port 9000), then run for 30+ minutes under light load.

# Test script (run from SSH for each value):
test_heap() {
  local size=$1
  echo "Testing --max-old-space-size=$size..."
  export NODE_OPTIONS="--max-old-space-size=$size -r /root/hijack.js --expose-gc --no-warnings"
  timeout 120 node /root/.openclaw/node_modules/openclaw/dist/cli.js gateway run --port 9000 &
  local pid=$!
  sleep 30
  if curl -s http://localhost:9000/api/status > /dev/null 2>&1; then
    echo "  $size MB: BOOT OK (pid $pid, RSS $(ps -o rss= -p $pid) KB)"
    kill $pid 2>/dev/null
  else
    echo "  $size MB: FAILED"
  fi
  wait $pid 2>/dev/null
}

Results from the binary search:

The binary search narrowed it to the 128-192 range. Within that range, 192 was the lowest value that provided consistent multi-hour stability. 160 would sometimes survive boot but OOM under load when a large API response triggered allocation.

Verification

# Confirm the minimum viable heap runs stable:
export NODE_OPTIONS='--max-old-space-size=192 -r /root/hijack.js --expose-gc --no-warnings'
node dist/cli.js gateway run --port 9000 &

# After boot, check RSS:
ps -o pid,rss,comm -p $(pgrep -f openclaw-gateway)
# RSS ~175 MB regardless of heap cap (V8 native overhead is fixed)

# Run a sustained test:
while true; do
  curl -s http://localhost:9000/api/status > /dev/null && echo "OK $(date)" || echo "FAIL $(date)"
  sleep 60
done
# Should print OK indefinitely at 192 MB

Gotchas

• RSS does not scale linearly with heap cap. At 192 MB cap, RSS is ~175 MB. At 384 MB cap, RSS is ~260 MB. V8 reserves virtual memory based on the cap but only commits pages as needed. However, the reserved pages still contribute to RSS on Android
• The minimum changes when the runtime stack changes. Moving from proot to native node22-icu shifted the minimum from 192 MB down to 112 MB because proot's syscall interception added memory overhead during module compilation
• Boot is the hardest phase. If a heap cap survives startup, it almost always survives steady-state operation. The exception is large API responses (multi-MB JSON payloads) which can spike allocation
• Each failed test requires a full process restart. There is no way to increase the heap cap at runtime
• The binary search took approximately 2 hours including reboot cycles and stabilization waits

Result

The binary search methodology proved its value repeatedly. Every major stack change (proot removal, lazy loading, native build) required re-running the search. The method is simple, deterministic, and gives you confidence in the result. The 192 MB finding directly led to Hack #20 (establishing 192 as the production minimum) and Hack #21 (further tuning semi-space on top of the established heap cap).