Blob: a null device part is an empty part

Merged
RO
1mo ago
+91-10

Follow-up to #33600, stacked on its branch. Related to #25851.

Problem

  • On the head of #33600, new Blob([Bun.file("/dev/null"), "x"]) throws TypeError: Blob parts backed by a pipe, socket or device cannot be read synchronously; await .bytes() or .arrayBuffer() first. main and Node v26.3.0 give a Blob of size 1.
  • push_file_part (src/runtime/webcore/Blob.rs) refuses each part that is not a regular file. That includes the null device.

Fix

  • A null device part is an empty part. A path is still not opened.
  • A part is the null device when it is a character device with the device number of the null device: major 1, minor 3 on Linux, the st_rdev of /dev/null or \\.\NUL elsewhere. This covers a path, an fd and a symbolic link.
  • Verified: test/js/web/fetch/blob.test.ts on Linux and on a Windows x64 build. The 2 new tests fail on the head of #33600 and on bun 1.4.2.

Background

  • st_rdev is the stat field that names the device behind a device file.
  • #33600 reads a file part in the constructor. A pipe or a device can block that read or never end, so it is refused. The null device gives end of file at once.
  • The first commit has the optional tests from the review of #33600.

Downsides

  • A character device part costs one compare on Linux and one more stat elsewhere. No other part reaches that code.
  • /dev/zero and a /dev/stdin that is a pipe still throw. Node gives an empty part for both.
  • Binary size: bun +0 bytes, bun-profile -3,440 bytes.
Notes

Results (release builds, Linux x64)

Node takes the length of a part from the stat size, so /dev/zero and a pipe are an empty part there. bun throws for them so that no bytes are dropped without a signal.

part of new Blob(["x", part])mainhead of #33600this PRNode v26.3.0
Bun.file("/dev/null")size 1TypeErrorsize 1size 1
Bun.file(fd) on /dev/nullsize 1TypeErrorsize 1
a symbolic link to /dev/nullsize 1TypeErrorsize 1
Bun.stdin with < /dev/nullsize 1TypeErrorsize 1
Bun.file("/dev/zero")size 1TypeErrorTypeErrorsize 1
/dev/full, /dev/urandom, /dev/ttysize 1TypeErrorTypeError
Bun.stdin as a pipesize 1TypeErrorTypeError
a regular file of 3 bytessize 1size 4size 4size 4

main gives size 1 in each row because it drops each file-backed part. That is the bug that #33600 corrects. For the null device the result was right by accident.

Windows (debug build of main plus this branch)

Bun.file("/dev/null"), NUL, nul, \\.\NUL, os.devNull and an fd on os.devNull are an empty part. CON, \\.\CON and CONIN$ throw the TypeError. Bun.stdin as a pipe throws the TypeError. libuv gives st_rdev = FILE_DEVICE_NULL << 16 for the null device and other values for a console and a pipe.

Linux

The kernel fixes the null device at major 1, minor 3 (Documentation/admin-guide/devices.txt). So the check needs no path. A process that has no /dev in its root, with an fd on the null device from its parent, gets an empty part too. I could not run that case: chroot is not permitted in the build container.

Tests

  • bun bd test test/js/web/fetch/blob.test.ts (debug, ASAN): 122 pass, 1 skip (the EACCES test does not run as root). As uid 65534 the block gives 13 pass.
  • With src/ of the head of #33600: "a null device part is an empty part" and "only the null device is an empty part" fail with the TypeError. The other tests pass.
  • With bun 1.4.2, which drops each file part: the same 2 tests fail, because each of them has a regular file as one more part.
  • bun scripts/rust-check-all.ts for aarch64-apple-darwin, x86_64-pc-windows-msvc, aarch64-linux-android and x86_64-unknown-freebsd: 4 ok. cargo clippy -p bun_runtime has no finding in Blob.rs.
  • Windows x64 debug build, the block of #33600: 9 pass, 4 skip.
  • The /dev/zero case runs in a child process. If the rule became too wide, a read of /dev/zero in the test process would not return.

In-memory parts

The two call sites in from_js_without_defer_gc are not changed. The new code is in push_file_part, which is #[cold], and in is_null_device.

When #33600 merges

I will rebase this branch on main and set the base of this PR to main.

no test proof · iteration 11 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/fetch/blob.test.ts