fix: Fallout 2 download and a transient manifest-code failure no longer dropping a depot

Merged
JO
1d ago
+978-67

Description

Fallout 2 could not be installed at all. Its packages grant Fallout 2 Depot New Base (38414) and Depot New English (38415), and the two disagree about what master.dat is: 38414 declares it as a directory, 38415 ships it as a 317.74 MiB file. A case-insensitive volume can hold only one of them, so whichever depot is laid down second fails — mkdir returns EEXIST over the file, or the file's open returns EISDIR over the directory — and the depot takes the whole run with it. (The pre-2020 depot 38411, which older appinfo still lists, has the same name in upper case as a file.)

The file wins. After every manifest of the run is resolved, and before anything else consumes them, a path that is a file in one depot and a directory in another loses the directory — together with everything under it, in every depot, since a child of that path could not be written anyway — and it is logged:

depot-clash path=master.dat kept=file depot=38415 dropped=directory depot=[38414]
depot-clash depot=38414 dropped N entries under a directory a file shadows

A directory is only a container, the file is the form the game can read (the Fallout engine opens master.dat as an archive), and it is what a Windows install converges on — Valve's own client cannot create the directory over the file either. Paths only clash when they fold to the same name, so a run with no clash is untouched. The probe sample, the removed-content sweep's ownership set and every write plan see the reduced manifests, so nothing downstream tries to place a dropped entry.

The write path applies the same rule to what is already on disk, so an install left in the wrong shape by an earlier attempt repairs itself instead of failing: a directory in a file's way is removed once the free-space preflight has passed (so a failed preflight leaves the old tree instead of deleting it), a file in a directory's way is unlinked, and a symlink entry clears either — a symlink that resolves to a directory is left alone, since it may be another depot's entry. Depots the store already records as installed take part in the decision even when the press does not select them, so a run can never replace a file an installed depot owns, and a file the delta calls unchanged is only trusted while it is still the regular file of the manifest's size on disk, so a truncated, missing or directory-shadowed path is revalidated rather than skipped.

Verified on device: Fallout 2 installs; master.dat lands as a ~317.7 MiB file with no master.dat/ folder, and the game launches.

A manifest request code that could not be fetched no longer costs a depot. The lookup swallowed every exception and returned 0, so one transient RPC failure marked the depot unavailable — and unavailable depots are left out of the run, which could then finish as installed without their files. It now answers three ways: a code from the first app that issues one; 0 only when Steam answered 0 for every app asked (skip it, with the reason — every host would answer 401); and nothing when a request failed and no app produced a code, in which case the depot is still tried without a code, because a manifest already on disk needs none and a fetch that does need one fails with the real auth error instead of the depot quietly missing. The 401/403 recovery order is a tested state machine as well: the request code is refreshed at most once per fetch, before any host is written off, then a host token once per host, and only then does the host count as rejecting auth.

cargo test --release 264/264, zero warnings — the plan-time pass (an exact folded-path clash drops the directory and its subtree in every depot, including an upper-case copy, while a non-clashing folder like sound/music is untouched; children of a file are dropped even when no directory entry declares the parent; an installed depot's file is protected), both disk transitions (a file replaces a directory that has contents; a directory replaces a stale file), the trusted-shape gate (intact, truncated, directory, missing), and the symlink rules (a symlink to a directory inside the tree is kept, one that resolves outside it is replaced, one standing where a file must go replaces it, a dangling one is cleared, a removal that would leave the install dir through a symlinked ancestor is refused, and a directory the manifest never declares is refused when its ancestor is such a link). ManifestRequestCodeTest (10 tests) covers the code-lookup decision and its lazy candidate loop under testLegacyDebugUnitTest. compileLegacyDebugKotlin + compileLegacyDebugJavaWithJavac clean; libgndownload.so rebuilt for both ABIs.

Recording

Type of Change

  • Bug fix
  • Performance / stability improvement
  • Compatibility improvements
  • Other (requires prior approval)

Checklist

  • If I have access to #code-changes, I have discussed this change there and it has been green-lighted. If I do not have access, I have still provided clear context in this PR. If I skip both, I accept that this change may face delays in review, may not be reviewed at all, or may be closed.
  • This change aligns with the current project scope (core functionality, stability, or performance). If not, it has been explicitly approved beforehand.
  • I have attached a recording of the change.
  • I have read and agree to the contribution guidelines in CONTRIBUTING.md.

Summary by cubic

Fixes Fallout 2 installs failing when two depots disagree about what master.dat is: "Fallout 2 Depot New Base" (38414) declares it a directory, "Depot New English" (38415) ships it as a 317 MiB file. On a case-insensitive volume only one can exist, so whichever depot is written second fails with EEXIST or EISDIR and aborts the run.

The file wins. After all manifests are resolved, a directory shadowed by a file of the same folded path is dropped — together with its subtree, in every depot — and logged (depot-clash). Installed depots join the decision so a run never replaces a file an installed depot owns. The write path applies the same rule on disk, so an install left in the wrong shape by an earlier attempt repairs itself instead of failing. Runs without a clash are untouched.

Also fixed

  • A transient manifest-code lookup failure no longer drops a depot from the install: 0 is concluded only when Steam answered 0 for every app, and a failed request leaves the depot tried without a code.
  • Every creation and removal now checks that the nearest existing ancestor resolves inside the install root, so a symlink anywhere above a path — including a directory the manifest never declares — cannot redirect the run.
  • A symlinked directory entry is kept only while it resolves inside the install tree; one pointing outside is replaced by a real directory.
  • A file the delta calls unchanged is only trusted while it is still the regular file of the manifest's size on disk; truncated, missing, or directory-shadowed paths are revalidated.

Summary by CodeRabbit

  • Bug Fixes
    • Installs now handle conflicting file and directory paths consistently across depots, keeping the file and skipping conflicting directories and their contents.
    • Existing installs can be repaired when file and directory paths conflict. Files are treated as unchanged only when they remain regular files with the expected size.
    • Install repairs handle conflicting symlinks without following paths outside the install directory, while retaining directory symlinks that point within it.
    • Manifest downloads distinguish confirmed unavailable responses from failed lookups, helping downloads proceed when a request fails. Failed authorization can retry with a refreshed, different request code.
  • Documentation
    • Updated documentation to explain path-conflict handling and install safety checks.