#44535 ws: take the frame type of send() from options.binary and frame the payload natively

The reference is node v26.3.0 with ws 8.18.3, the version that test/package.json pins. send() and toBuffer() are the same code in ws 8.21.0. "base" is the head of #44520 (a87f51638b, on main 519963edc8) and "PR" is the head of this branch, both as release builds from one build directory. Raw TCP peers with no websocket implementation read the frames.

The rule of npm ws (websocket.js#L448-L478, sender.js#L346-L363)

if (typeof data === 'number') data = data.toString();
const opts = { binary: typeof data !== 'string', mask: !this._isServer, compress: true, fin: true, ...options };
this._sender.send(data || EMPTY_BUFFER, opts, cb); // opcode = opts.binary ? 2 : 1

This PR has the frame type for a string, a number, bytes and a Blob. It does not have data || EMPTY_BUFFER and toBuffer(data) for any other value. That payload rule is row 16 of #44108, and the same rule at ping() and pong() is row 26. An earlier shape of this PR had it for send(). It left, because it is a second change for callers that pass no binary (send(true) throws), and because a client socket in fragments mode hands a text message out as an array of one Buffer: with the rule, the README echo of that message went out as one 00 byte.

The one change that a peer can reject: bytes that are not UTF-8 in a text frame

// node ff68.mjs, bun ff68.mjs
import { WebSocketServer, WebSocket } from "ws";
import net from "node:net";
const wss = new WebSocketServer({ port: 0, host: "127.0.0.1" });
await new Promise(r => wss.on("listening", r));
const port = wss.address().port;
const invalid = Buffer.from([0xff, 0x68]);
const raw = await new Promise(resolve => {
  wss.once("connection", ws => ws.send(invalid, { binary: false }));
  let buf = Buffer.alloc(0);
  const peer = net.connect(port, "127.0.0.1", () =>
    peer.write("GET / HTTP/1.1\r\nHost: x\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\nSec-WebSocket-Version: 13\r\n\r\n"));
  peer.on("data", chunk => {
    buf = Buffer.concat([buf, chunk]);
    const end = buf.indexOf("\r\n\r\n");
    if (end === -1 || buf.length < end + 6) return;
    peer.destroy();
    resolve(buf.subarray(end + 4).toString("hex").replace(/../g, "$& ").trim());
  });
});
const events = await new Promise(resolve => {
  const seen = [];
  let open = 2;
  const done = () => void (--open === 0 && resolve(seen));
  wss.once("connection", ws => {
    ws.on("error", e => seen.push(`server error ${e.code ?? e.message}`));
    ws.on("close", code => (seen.push(`server close ${code}`), done()));
    ws.send(invalid, { binary: false });
  });
  const client = new WebSocket("ws://127.0.0.1:" + port);
  client.on("message", (data, isBinary) => seen.push(`client message ${isBinary ? "binary" : "text"} ${Buffer.from(data).toString("hex")}`));
  client.on("error", e => seen.push(`client error ${e.code ?? e.message}`));
  client.on("close", code => (seen.push(`client close ${code}`), done()));
  setTimeout(() => resolve(seen), 2000).unref?.();
});
console.log(`frame on the wire: ${raw}\na ws client that gets it: ${events.join(", ")}`);
process.exit(0);

node v26.3.0 + ws 8.18.3
  frame on the wire: 81 02 ff 68
  a ws client that gets it: client error WS_ERR_INVALID_UTF8, server close 1007, client close 1006
this PR
  frame on the wire: 81 02 ff 68
  a ws client that gets it: client close 1007, server close 1006
base
  frame on the wire: 81 04 ef bf bd 68
  a ws client that gets it: client message text efbfbd68

A sender that relied on U+FFFD to make bytes valid now gets its connection failed by a peer that validates. npm ws gives a sender no validation either.

Wire: 85 rows on both sockets, 170 cells

The 40 cells of the last line have the same output on the base and on the PR.

What changes for a caller. Each line is what npm ws does.

• send(ArrayBuffer | DataView | SharedArrayBuffer | Blob, { binary: false }) is a text frame. Before: a binary frame, and on a client socket ERR_INVALID_ARG_TYPE in the callback for a SharedArrayBuffer or a resizable ArrayBuffer.
• send(view, { binary: false }) sends the bytes as they are. Before: U+FFFD for each byte that is not UTF-8.
• Only the truth of an own binary counts: 0, null, '', NaN and an own undefined give a text frame, 1, 'yes' and {} a binary frame. A binary that options inherits does not count. Before: only true and false counted, and an inherited one counted.
• A message that waits in the queue of a server socket goes out with the frame type of its send().

Where send(data, options) is not npm ws after this PR

The exact rule for an own binary that is not enumerable takes a propertyIsEnumerable() call for each frame whose binary differs from what the type of the data gives. That frame is the text echo. The call costs 277 instructions, and the options literal of the caller is then allocated. Only Object.defineProperty() with enumerable: false makes such a property. main reads options.binary too, so it has this difference for a Buffer with false and a string with true. For other shapes main is right by accident: 4 of the 14 cells are right on the base and wrong here (an ArrayBuffer with 0, a string with 1). If the exact rule is wanted, it is three lines in isBinaryFrame().

A client socket that is not open. One value changes: a Blob after close() adds 10 to bufferedAmount, as a string and a Buffer of 4 bytes do (the payload, and 6 for the frame head and the mask). On the base it adds 0. npm ws adds 4. While CONNECTING nothing changes: the callback gets InvalidStateError, where npm ws throws. #39840 has that difference. Two tests hold this path.

ping() and pong() are not touched.

The stall of the queue of #44520 that this PR ends

The second commit of #44520 puts a send() behind the messages that wait, with no call of the native socket. A value that the native socket rejects then throws in each drain and stays at the head of the queue. The review thread at ws.js:1091 of #44520 reports it. Here the entry checks data and compress at the call, so the value does not reach the queue. Probe: a peer that does not read, send() of 4 MiB until one message waits, the call, an end mark, then the peer reads.

The send() of main calls the native socket before it looks at the queue, so on main these calls throw at the call too. If #44520 merges alone, main has this stall until this PR merges. A try/catch around the send in #drain of #44520 would close that gap. This PR then removes it.

Compression on a client socket. The native client deflates a frame that does not fit its stack buffer of 1,024 bytes, which is a payload of 1,016 bytes or more. For a UTF-16 string of 256 characters or more the limit is 860 bytes. The entry gets the data in its own kind. So between 860 and 1,015 bytes, bytes that are not ASCII sent as a text frame are no longer deflated (the base made a UTF-16 string of them), and a UTF-16 string sent as a binary frame now is (the base made bytes of it). npm ws has one limit, 1,024 bytes. One test reads RSV1 and inflates both frames at 2,048 bytes.

Instructions for one send

A ptrace tracer single-steps the main thread between two marker calls around one statement, after 20,000 runs of that statement. BUN_JSC_useConcurrentJIT=0, median of 21 samples. Each count includes 58 instructions of the markers. A peer in the same process sends one message for each run. The counter does not see the kernel, other threads, or the samples that the median drops (a GC, a page fault). The time tables below do.

Public Bun.serve handler, 16-byte payload unless said. Base and PR are equal in all 18.

Public WebSocket client: send(string) 1,070, send(buffer) 1,096, send(arrayBuffer) 812, send(blob) 866 and the two 1 KiB sends are equal. ping() and pong() go from 1,491 to 1,484 with the layout of the binary: in other builds of this branch the 7 were on other rows. The 35 public send, publish, ping, pong and cork functions have the same size and the same instructions in both binaries (objdump, addresses taken out). 6 instructions differ in one operand, the address of a table that moved by 64 bytes.

Socket of the ws module, base to PR:

The counts of the PR are the same in all 21 samples for most rows. The counts of the base move between samples where the base allocates: send(buffer, { binary: false }) on a server socket is 2,092 at best and 17,998 in the median of this run.

The last two client rows take more instructions and less time (see Time). The native client writes a string with one length pass more than Buffer.from() plus a masked copy. A text send(string) of the same 16 KiB string takes 19,031 on the base and 19,007 here.

A send() on a server socket behind a message that waits, base to PR: a 16-byte string 435 to 563, a Buffer 170 to 399, a Blob 217 to 463, a 1 KiB string 640 to 757, a 16 KiB string 3,880 to 4,008, the text echo 2,180 to 417. The plain sends pay for the call of the entry that checks the value. The base did not check it, which is the stall above.

Time

Two release builds, runs in pairs of base and PR, median with the range of the runs. The machine has 12 cores and ran nothing else.

A send(string) behind a message that waits, 3 pairs, ns for each call: 16 bytes 203 and 218, 1 MiB of ASCII 28,185 and 29,882, 1 MiB of Latin-1 28,414 and 29,014, 1 MiB of UTF-16 111,321 and 115,628. The ranges of base and PR overlap or touch in each row.

bench/websocket-server/chat-server.node.mjs (the ws server, 32 clients in 4 processes, plain send(string)), 8 pairs with the order swapped in every other pair: 110,924 messages for each second on the base (99,683 to 125,393) and 121,420 on the PR (105,317 to 135,422). CPU time of the server for each message: 5.31 µs (4.50 to 5.83) and 5.41 µs (4.66 to 6.60). No difference outside the spread.

The first sends of a process, before the JIT reaches its last tier (instructions, base / PR):

The server numbers at 20,000 sends need #frame to reach the last tier before the functions that call it. The last tier inlines a function of this size only after that function has its own code from that tier. In an earlier shape of this PR the queue code was inside #frame, it reached the tier after its callers in this benchmark, it stayed a call, and each server send took 49 instructions more. The queue code is in #enqueue, as in #44520.

Considered a split of isBinaryFrame() that the first optimizing tier can inline: it takes 45 instructions off a plain server send(buffer) at 5,000 sends and puts 54 to 67 on each send with a binary at 20,000. Not taken.

JS cells for each send in the last tier (heapStats(), BUN_JSC_useGC=0): 0 for all 9 shapes on both sockets. Base: a Uint8Array and a string for send(buffer, { binary: false }), a Uint8Array for send(string, { binary: true }). mi_malloc calls in one send: none in the 10 shapes traced. Base: 1 to 6 in 9 of them.

Size of the stripped binary: text 80,702,607 to 80,706,325 bytes. New code: jsSendFrame 1,450 bytes, jsWebSocketSendFrame 1,075, WebSocket::sendFrame(JSBlob*, Opcode) 590.

The queue of the server socket

The queue is the one of #44520: #enqueue, the byte count, the bufferedAmount getter, and the rule that a send() behind messages that wait goes to the end of the queue. This PR changes three things in it.

• Each entry keeps the frame type of its send(), and #drain sends it through the frame entry.
• The entry returns false for a frame that did not go out. That replaces wasDropped(), which asked the socket for its buffered amount to tell a dropped frame from an empty one.
• A send() behind the queue calls the entry with no socket. The entry then sends nothing. It checks data and compress, and it does not encode a string. A socket that is not open yet has no native socket either and takes the same route.

The public send() return value does not change. #35238 changed it and was closed.

Order with other open PRs

• #44520 is the base. This PR cannot merge before it. When it merges, this branch rebases on main. The two belong in one release, because of the stall above.
• #39802 has the close path of the server socket and send() after close. It conflicts with #44520 in ws.js already. It rewrites the same lines of send() and #drain. The one that merges second keeps the frame entry in them.
• #44112 changes the decode line of normalizeData. This PR deletes the function. The second to merge drops that line. #44112 and #44111 add the import of the npm server module that this PR adds to ws.test.ts.
• #39840 wraps the client send() call in state checks. The call inside becomes the frame entry.
• #35031 adds a helper above normalizeData in ws.js.
• #44111 gives on('message') of a server socket a Buffer for a text frame in every binaryType. #44255 and #44553 do that for a client socket, where a text message in fragments mode is an array of one Buffer today. They change what the README echo passes to send(). This PR makes the send right for a string, bytes and a Blob.
• #35459 calls ws.send(chunk, callback) on a ws socket, outside the two socket classes. The lint does not look there.
• #35029 (detached buffers), #39082 (Blobs in a file) and #31761 (bufferedAmount after close) change the public send() of the native client. The one that merges second makes the same change in sendFrame().
• #44618, #44554, #44255, #44553, #39093 and #39370 touch the same files and merge with this branch with no conflict (git merge-tree).

Tests. The raw peer of the new tests is in test/js/first_party/ws/raw-peer-test-utils.ts. Each wire test runs on the built-in module and on npm ws, so each expectation is the output of npm ws. Of the new block, 9 tests fail on the base: the four tables, the echo of a server socket in arraybuffer and blob mode, the queue test, the test of a client socket that is not open (the Blob), and the test of a binary that is not enumerable. The queue test asserts that a message waits, but not on Windows: node-http-server-close-drain.test.ts and websocket-server-backpressure-buffer.test.ts say that Winsock takes the whole payload from a peer that does not read.

Lint. test/internal/source-lints/ws-frame-entry.test.ts fails when one of the two socket classes in ws.js calls send(), sendText() or sendBinary() on a native socket. ws.js is now in the paths of source-lints.yml.

Self-review. Three rounds. The first one: the overlap with #39802 and #35238 was not named, the lint matched every .send( in the file, the text claimed more than the diff does, the own binary that is not enumerable was described as "as on main", no node output for the invalid UTF-8 change, a wrong close code in a comment, #44111 was not named. All addressed. The second one, 13 concerns: this PR carried its own version of the queue code of #44520 (now the base), the #open reorder and the test with a socket made by hand (both removed), no test of the client entry on a socket that is not open (two tests added), the description did not name #44520. The third one, 19 concerns, 17 addressed: the payload rule for other values was a second change and broke the echo in fragments mode (removed, with its tests), a string behind the queue was encoded and thrown away (the entry answers first), the cost section had no time (added), the raw peer helpers (now a module), tests that did not fail for a socket that closes early (the helpers reject on 'close'), the deflate test did not read the wire (it reads RSV1 and inflates), no row for an 8-bit string over 0x7f (two rows), the queue test passed with no message that waits (asserted), the CONNECTING test took any Error (it names the channel and the error), no test for compress and a file Blob behind the queue (two lines), and the order list (updated). Not taken, 2: a try/catch in #drain of #44520 and a move of the generic half of the queue test into #44520. Both are changes to that PR. Its thread has the probe and the proposal.

Found outside this PR

• The native client writes a UTF-16 string with two length passes (Copy::frame_and_content_len, then copy_utf16_into_utf8_impl in src/http_jsc/websocket_client.rs). One pass less makes the public WebSocket.send(string) faster for large strings.
• A client socket that gets a text frame with invalid UTF-8 emits 'close' with 1007 and no 'error', and sends no close frame. node emits 'error' with WS_ERR_INVALID_UTF8, then 'close' with 1006, and the peer gets a close frame with 1007. Row 23 of #44108.
• uWS drops a frame only under a backpressureLimit that is not 0 (packages/bun-uws/src/WebSocket.h:139). The websocket block of the node:http bridge leaves the default of 16 MiB on. With 0 the queue of the server socket has nothing to keep.

Tests run

• Debug build with ASAN: test/js/first_party/ws/ws.test.ts 115 pass, 2 skip, 5 todo, 0 fail. The new block under BUN_JSC_validateExceptionChecks=1: 36 pass.
• Release builds: test/js/first_party/ws/ 147 pass on the PR, 138 pass and 9 fail on the base. test/js/bun/websocket/ 183 pass on both. test/js/web/websocket/ 275 pass and 2 fail on both (the 2 need the public internet).