shell(mkdir, touch): report operands longer than the path buffers instead of aborting

Merged
RO
3mo ago
+164-8

Problem

  • Bun.$ mkdir <operand> with a relative operand longer than the join buffer, and touch <operand> with any operand longer than a path buffer, abort the process: panic: range end index 5004 out of range for slice of length 4094 (cwd /tmp, 5000-byte operand; same for mkdir -p and for an absolute touch operand).
  • mkdir with an absolute operand that does not fit a path buffer reports the wrong error: mkdir: /aaa...: No such file or directory instead of File name too long.
  • Cause, crash: ShellMkdirTask::run_from_thread_pool (src/runtime/shell/builtin/mkdir.rs) joins a relative operand onto the cwd with resolve_path::join_z, which writes into a fixed 4096-byte thread-local buffer; ShellTouchTask::run_from_thread_pool (touch.rs) joins both kinds of operand with join_z_buf into a stack PathBuffer. Neither join bounds-checks its output (normalize_string_generic_tz in src/paths/resolve_path.rs), and the operand comes straight from the user.
  • Cause, wrong error: mkdir hands the joined path to the node:fs mkdir implementation as a PathLike. For JS callers, Valid::path_string_length (src/runtime/node/types.rs) rejects paths of MAX_PATH_BYTES or more up front; for anything longer, PathLike::slice_z returns "" and mkdir("") fails with ENOENT. The shell builds the PathLike itself and skipped that check.

Fix

  • Both builtins join through join_z_spill, the existing variant that falls back to a caller-owned Vec when the parts would not fit (the same helper fs.readdir, Bun.Glob and the open rm fix #37521 use). The result is the same normalized path as before for every operand that used to work.
  • mkdir then reports ENAMETOOLONG itself, naming the path, when the path it is about to create is MAX_PATH_BYTES or longer, i.e. the bound node:fs assumes. For a relative operand that is the joined, normalized path, so a long ././.../x spelling is still created; an absolute operand is passed on as written (as before: normalizing it would change what .. means across a symlink), so it is bounded as written, which is also what the kernel and coreutils do with such an operand. Both cases are in the test.
  • touch needs no check of its own: it calls bun_sys::utimens / open with the string as is, and the OS reports ENAMETOOLONG for it like for any other operand. It also no longer puts a PathBuffer on the worker's stack (about 96 KB on Windows).
  • Verified with test/js/bun/shell/commands/mkdir.test.ts and touch.test.ts (the commands/ directory has one file per builtin; these two had none). Each runs the builtin in a child bun and covers a 5000-byte relative and absolute operand, mkdir -p, a 100000-byte operand (longer than the buffer on Windows too), a long operand next to a normal one that must still be created, and a 6000-byte ./ spelling that must still succeed (for mkdir also the absolute form of it, which must be refused as written; POSIX only, since on Windows it fits the buffer).
    • USE_SYSTEM_BUN=1 bun test test/js/bun/shell/commands/mkdir.test.ts test/js/bun/shell/commands/touch.test.ts: both fail, the child aborts with the panic above. Without the new mkdir check, the mkdir cases fail on the ENOENT message instead.
    • bun bd test test/js/bun/shell/commands/mkdir.test.ts test/js/bun/shell/commands/touch.test.ts: pass. bunshell.test.ts, file-io.test.ts, commands/rm.test.ts pass as well; cargo check for the Windows and macOS targets is clean.
    • Windows: the released build aborts on the 5000-byte relative mkdir operand there too (the join buffer is 4096 bytes on every platform), and creates the directory for the absolute ././.../x spelling (the branch this change does not touch below 98302 bytes), which is what the Windows side of the test asserts; both checked on a Windows x64 machine. The Windows lanes of this PR's CI runs pass both test files.
  • Related open PRs: rm (#37521), mv (#37527) and cp (#38162) fix the same pattern in their builtins; #38162 adds shell_join_path helpers in interpreter.rs that touch could switch to in one line once either PR lands. #38002 (empty operands) also creates commands/mkdir.test.ts and touch.test.ts; whichever lands second appends its test to the other's file. ls -R joins real directory entries through the same thread-local buffer, but that needs an on-disk tree within PATH_MAX whose entries join past 4096 bytes rather than a long operand, and is left alone here.

Background

  • Shell builtins run in-process; mkdir and touch schedule one task per operand on the worker pool. A task either succeeds or stores a bun_sys::Error, which the builtin prints as <cmd>: <path>: <coreutils message> and turns into exit code 1 once every task has finished, so one failing operand does not stop the others.
  • The shell has its own cwd ($.cwd(), cd), which can differ from the process cwd, so these builtins make relative operands absolute by joining them onto the shell cwd as strings before calling an implementation that takes a path.
  • resolve_path::join_z and join_z_buf concatenate and normalize their parts (., .., repeated separators) into a fixed buffer without checking that the result fits; the *_spill variants take a Vec to grow into when the unnormalized length would not fit, and otherwise behave identically.
  • PathBuffer is [u8; MAX_PATH_BYTES]: 4096 bytes on Linux, 1024 on macOS, 98302 on Windows. The node:fs layer copies every path it receives into one, which is why its JS entry points reject longer paths before calling it.

no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/shell/commands/mkdir.test.ts test/js/bun/shell/commands/touch.test.ts