feat(scaling): engine.io WS transport-level packing — lever 8 prototype

Closed
JO
5mo ago
+114-0

Summary

Prototype for the engine.io transport-level packing direction from #7767 — the meatiest untouched lever in the scaling-dive doc.

What changes. engine.io's WebSocket transport sends one WS frame per engine.io packet, even when the engine.io socket has multiple packets buffered. The polling transport doesn't — its `transport.send(packets)` runs through `encodePayload` and writes one HTTP response containing the whole payload. This PR monkey-patches engine.io's WS transport so multi-packet flushes go out as ONE frame containing the payload-encoded bundle. Single-packet sends keep the legacy fast path (including the pre-encoded-frame optimisation), so steady-state quiet-pad behaviour is identical to upstream.

Why. At high emit rate the WS path is dominated by per-frame syscalls on the server and per-message callback overhead on the client. The dive measured this directly via lever 4 (`socketTransportProtocols: ["websocket"]`): dropping the polling fallback at 200 authors quadrupled client p95 and nearly tripled `apply_mean`. The polling fallback was acting as a natural coalescer; this PR brings the same coalescing property to the WS path.

Implementation

  • New module `src/node/utils/EnginePacking.ts` — installs the patch at boot. Idempotent. Resolves engine.io's own `engine.io-parser` instance so we use the exact `encodePayload` the transport already imports.
  • `src/node/hooks/express/socketio.ts` — calls the installer once, BEFORE constructing the socket.io `Server`, so the patched prototype is in effect when transports are created.
  • `src/node/utils/Settings.ts` — `enginePacking: false` default, documented warning about client compatibility.

Rollout warning

Enabling this on a server with clients that haven't been updated will silently break those clients: a multi-packet frame fed to engine.io-client's stock `decodePacket` returns garbage. The harness-side patch in ether/etherpad-load-test#TBD is forward-compatible (detects `\x1e` and routes through `decodePayload`). For production rollout the etherpad browser bundle would need the same forward-compat patch — out of scope for this prototype.

Scoring this

Once merged (or via `core_ref=feat/engine-io-ws-packing`):

```
gh workflow run "Scaling dive" --repo ether/etherpad-load-test --ref main \
-f core_ref=feat/engine-io-ws-packing \
-f sweep='authors=100..500:step=50:dwell=8s:warmup=2s'
```

The dive workflow includes a new `engine-packing` matrix entry that sets `enginePacking: true` in `settings.json`. Expectations:

  • `emits_NEW_CHANGES` count unchanged (we still emit N application-level messages).
  • WS frame count on the wire reduced proportionally to packets-per-flush.
  • Server CPU and `apply_mean` drop at the cliff, since per-frame syscall overhead drops.
  • Client-side perceived latency improves if the per-message callback overhead was a significant chunk of the wait.

Files touched

  • `src/node/utils/EnginePacking.ts` (new, ~50 lines incl. comments)
  • `src/node/hooks/express/socketio.ts` (+10 lines, conditional require)
  • `src/node/utils/Settings.ts` (+1 type field, +14 documented default)

No tests yet — the patch is best validated end-to-end via the dive workflow. A unit test that constructs an in-memory WebSocket transport and asserts the framing is feasible but high effort for a prototype; happy to add it before merge if reviewers want.

🤖 Generated with Claude Code