V8 Semi-Space 2 -> 1 MB

Context

V8 uses a generational garbage collector. Newly allocated objects land in the "young generation", which is divided into two equal-sized semi-spaces: one active (allocation space) and one idle (used during scavenging). When the active semi-space fills up, V8 performs a minor GC (scavenge): it copies surviving objects to the idle semi-space (or promotes them to old space if they survived a previous scavenge), then swaps the two spaces.

The default semi-space size on 32-bit ARM is 2 MB per space, meaning 4 MB total young generation. For a server-class Node.js app that allocates heavily (React rendering, JSON parsing), this makes sense. But the OpenClaw gateway is primarily I/O-bound: it proxies API calls to LLM providers, manages WebSocket connections, and streams responses. The allocation rate is modest, and most short-lived objects die quickly.

Shrinking the semi-space from 2 MB to 1 MB means minor GCs happen more frequently, but each scavenge is faster (less memory to scan). On a single-core Snapdragon 410, the total GC time stays roughly the same while RSS drops.

Implementation

Add the flag to NODE_OPTIONS in the start script:

# In $PREFIX/bin/start-openclaw:
export NODE_OPTIONS='-r $HIJACK --expose-gc --no-warnings --max-old-space-size=150 --max-semi-space-size=1'

The combined heap tuning settings tested during this phase:

The semi-space flag interacts with the old-space cap. Smaller semi-space means more objects get promoted to old space sooner, increasing old-space pressure. The 150 MB old-space cap provides enough headroom for this.

Verification

# Check V8 heap stats from inside the gateway (via --expose-gc):
curl -s http://localhost:9000/api/status | python3 -c "
import sys, json
d = json.load(sys.stdin)
heap = d.get('memory', {})
print(f'Heap used: {heap.get(\"heapUsed\", 0) / 1024 / 1024:.1f} MB')
print(f'Heap total: {heap.get(\"heapTotal\", 0) / 1024 / 1024:.1f} MB')
"

# Check RSS of the gateway process:
ps -o pid,rss,comm -p $(pgrep -f openclaw-gateway)
# Expected: RSS ~180 MB (down from ~216 MB)

# Monitor minor GC frequency (trace GC output):
NODE_OPTIONS='--trace-gc --max-semi-space-size=1' node -e "
  for (let i = 0; i < 10000; i++) Buffer.alloc(1024);
  console.log('done');
" 2>&1 | grep Scavenge | wc -l
# More scavenges than with default, but each is faster

Gotchas

• Setting --max-semi-space-size=1 causes significantly more frequent minor GCs. In later testing, this was refined to --max-semi-space-size=2 as the sweet spot. At 1 MB, the scavenge frequency on the gateway was roughly 3x higher, which added CPU load during LLM streaming
• The value is per semi-space, not total. --max-semi-space-size=2 means 4 MB total young generation (2 MB active + 2 MB idle)
• This flag has no effect if V8 decides to use a larger semi-space internally during startup. The flag sets the maximum, not the exact size
• On the native node22-icu build (Hack #12), the final tuned value became --max-semi-space-size=2 with --max-old-space-size=112. The 150 MB old-space cap from this hack was later reduced as heap profiling improved
• Do not confuse with --max-old-space-size. Semi-space controls young generation only

Result

The --max-semi-space-size=1 setting was the first version deployed and proved stable for 11+ hours. It was later relaxed to 2 to reduce minor GC churn on the single-core CPU. Combined with --max-old-space-size=112 on the native build, the final production NODE_OPTIONS line became:

export NODE_OPTIONS='-r $HIJACK --expose-gc --no-warnings --max-old-space-size=112 --max-semi-space-size=2'