Bun.write: don't cap a file-to-file copy at the destination's cached size

Closed
RO
2mo ago
+167-6

Problem

  • Bun.write(Bun.file(dest), Bun.file(src)) copies only as many bytes as dest used to have, once anything has read dest's size (.size, exists(), toHaveLength(), structured clone). With the usual exists() guard on a missing file, the copy leaves an empty file behind.
  • Only file to file is affected; string, buffer and Blob sources never look at the destination's size. Reproduced on 1.4.0 on Linux; the macOS and Windows copy paths consume the same value, and Windows also padded a shorter copy back out to the old length.
  • Cause: a file-backed blob keeps one size field for two things, a slice() window and the cached stat size. The copy treated whatever was in it as a window, so a destination that had merely been stat'ed was written as if it were dest.slice(0, oldSize).
  • Fixes #4930
  • Fixes #22456

Fix

  • The blob now records whether its size came from slice(). The copy is bounded by the destination only in that case; otherwise it is bounded by the source, as a fresh Bun.file(dest) or a path destination already was.
  • Structured clone puts a size on the wire only for sliced blobs and treats a received size as a window, so a clone of a stat'ed whole-file blob stays unbounded and a clone of a real slice keeps its window. No wire format change.
  • A destination the caller sliced behaves exactly as before; only the cases that lost data change.
  • Verification: new tests prime the destination each of the six ways, plus the missing-file guard, an fd destination, a shrinking copy and sliced destinations. 14 of 17 fail on the release bun, all pass with the fix. The returned byte count is asserted only on Linux; macOS and Windows resolve with 0 over an existing file regardless of this change (#33715).

Background

  • Bun.file(path) is a lazy blob: it holds a path or fd and does not stat the file until asked. .size, exists(), toHaveLength() and structured clone all stat it and cache the size on that blob object.
  • blob.slice(start, end) is a new blob over a byte window of the same file. As a Bun.write destination it means "write into these bytes only", and existing tests rely on it capping a copy.
  • Before the stat, the size field holds MAX_SIZE, the sentinel for unknown. The per-platform copy routines (Linux, macOS, Windows) take a max length, and MAX_SIZE there means the source decides; all three were being handed the destination's size field.
Original description

Bun.write(Bun.file(dest), Bun.file(src)) copies only as many bytes as dest used to have once anything has resolved dest's size. The common shape is an exists() guard, which leaves an empty file behind (#4930, open since 1.0); the other is overwriting a shorter file after exists() / .size (#22456).

Repro

import fs from "fs";
fs.writeFileSync("src.bin", Buffer.alloc(300000, "S"));
fs.writeFileSync("dest.bin", "short");

const f = Bun.file("dest.bin");
f.size;                                               // or: await f.exists()
console.log(await Bun.write(f, Bun.file("src.bin"))); // 5, expected 300000
console.log(fs.statSync("dest.bin").size);            // 5, expected 300000

const out = Bun.file("missing.bin");
await out.exists();                                   // false
await Bun.write(out, Bun.file("src.bin"));
console.log(fs.statSync("missing.bin").size);         // 0: the #4930 shape

Without the .size / exists() line both copies are complete. Only the file to file path is affected; string, buffer and Blob sources go through WriteFile, which never looks at the destination's size. Reproduces on 1.4.0 on Linux; the macOS (clonefile / fcopyfile + ftruncate) and Windows (CopyFileWindows::on_complete truncate) arms consume the same value. On Windows the cached size also works the other way round: copying a shorter file into a longer one padded it back out to the old length.

Cause

Blob.size holds two different things for a file-backed blob: the window set by slice(), and, once resolve_size() has run, the file's stat size. .size, exists(), expect(f).toHaveLength() and structured-clone serialization all run resolve_size() on the blob itself. write_file_with_source_destination (src/runtime/webcore/Blob.rs) then handed destination_blob.size to CopyFile::create / CopyFileWindows::init as the copy length, so a destination whose stat size merely got cached was copied into as if the caller had written Bun.file(dest).slice(0, oldSize).

Fix

Add Blob::size_is_explicit, set by slice() (and by dupe() / the other field-copy), and have the copy take the destination window only when it is set; otherwise it passes MAX_SIZE and the copy is bounded by the source like a fresh Bun.file(dest) or a path destination already is. slice() with no bounds on an unresolved file blob produces MAX_SIZE, which is not a window, so that case stays unbounded too.

Structured clone writes the v4 size field only for sliced blobs (the format already defines MAX_SIZE there as "receiver stats it itself") and sets the flag when it reads a concrete one. Without that, a clone of a primed whole-file blob would arrive looking like a slice of the old length, and a clone of a real slice would lose its window under the new check. No wire format change.

An explicitly sliced destination behaves exactly as before (bun-write.test.js already relies on Bun.file(fd).slice(0, 5) and Bun.stdout.slice(0, 100) capping the copy), so unlike the attempts that changed what a destination slice means, this only changes the cases that were losing data. It is also independent of which size ends up being consulted later: #31515 / #32859 (copy the source slice instead) would make the call-site hunk here redundant, and #33360 / #33659 add the same field for the read paths and the getters; whichever of those lands second has a one-line conflict on the field declaration.

Verification

test/js/bun/io/bun-write.test.js, new describe next to the existing file to file test: the six ways of priming the destination (.size, exists(), toHaveLength(), serializing it, writing to a clone of a primed blob, writing to a whole-file slice()) against a shorter and an empty existing file, the missing-destination exists() guard from #4930, an fd destination, a shorter source over a longer destination, and guards that slice(0, 4) and structuredClone(slice(0, 4)) destinations are still bounded. The resolved byte count is only asserted on Linux because uv_fs_copyfile and fcopyfile over an existing file resolve with 0 independently of this (#33715).

$ USE_SYSTEM_BUN=1 bun test test/js/bun/io/bun-write.test.js -t "not capped by the destination"
 3 pass, 14 fail
$ bun bd test test/js/bun/io/bun-write.test.js -t "not capped by the destination"
 17 pass, 0 fail

Also green with the debug build: the rest of bun-write.test.js (the subprocess tests there only time out when run concurrently under ASAN here, same as on main), structured-clone-blob-file.test.ts, blob.test.ts, blob-write.test.ts, blob-cow.test.ts, bun-file*.test.ts, bun-serve-file.test.ts and the serve.test.ts Content-Range suite. cargo check -p bun_runtime passes for x86_64-pc-windows-msvc.

Fixes #4930
Fixes #22456