Let the import privacy gate pass public technical forms without opening new leaks

Merged
UN
11d ago
+2066-47

Summary

The import privacy gate refused public technical text that names no person or machine, so scheduled issue syncs stopped (#193). It now allows exactly these forms:

  • Documentation email domains (RFC 2606/6761): example.com, example.net, example.org and localhost, matched exactly, plus any domain under .example, .invalid, .test or .localhost. Look-alikes such as notexample.com, example.com2 or example.com.evil.io are still refused.
  • The git SSH transport account in ssh:// and scp-style clone URLs. Any other SSH local part is still refused.
  • procfs and sysfs interface paths: the bare root, self, numeric pids and <placeholder> pids.
  • GitHub-hosted runner workspaces (POSIX and Windows) and the hosted tool cache.
  • Placeholder home roots: example, $USER, ${USER}, %USERNAME%, <…> and {…}.

Concrete accounts, root and temp trees, any other drive, ./.. and percent-encoded escapes, and identity roots embedded in a permitted tail all stay refused.

Review fix (5bf20e2). Angle and curly brackets became path-continuation characters, so that placeholder segments are judged whole. That let a refused path ride behind a permitted prefix: a non-placeholder segment ending in a bracket, followed by a root-account, temp, second-drive or tilde-username path, passed the gate. The previous gate refused these. hidesHostPathBehindBracket now treats such a bracket as the start of a new path and judges the rest with every host-path rule.

Evidence

  • Corpus: 45 must-allow and 54 must-refuse rows. Each runs as plain text, as JSON-escaped TOON and through a full synthetic tracker diff.
  • Mutation proof (node scripts/prove-gate-public-forms.ts): 20/20 mutations flip exactly their recorded rows. Removing any single allowance refuses its rows; broadening any shape admits its refusal rows. The new remove-bracket-smuggle-check mutation flips the 4 smuggled-* rows.
  • Real issue text: a read-only scan of 8 fleet repos (68 issues, 82 comments) compared findings before and after. pm-review dropped from 3 current findings to 0. Original snippets in pm-context, pm-rust and pm-swarm dropped to 0. The two pm-ops findings that remain are intended refusals of root or temp paths.
  • release:check: 1,609/1,609 tests and exact 100% coverage across 29 modules. Gate tests 134/134; changelog check passes. The object-store privacy gate audited 8,025 objects with no findings.

Full table and rule hashes: docs/gate-public-forms-evidence.md

The local coverage run used the same percentage heap override as #212. This PR's CI shows whether the checked-in 12 GiB cap holds on hosted runners (tracked in pm-github-vhi6).

pm items

Closes #193

Summary by Sourcery

Allow safe public technical forms through the import privacy gate while preserving strict protection against personal, host, and account-identifying data.

New Features:

  • Allow narrowly scoped public technical forms through the import privacy gate, including documentation-only email domains, git SSH transport URLs, procfs/sysfs interfaces, hosted runner paths, the hosted tool cache, and placeholder home roots.

Bug Fixes:

  • Prevent refused host paths from being hidden behind permitted prefixes or punctuation, including bracket, environment-placeholder, tilde, markup, and mixed-separator cases.
  • Preserve fail-closed handling for concrete identities, traversal and encoding escapes, embedded filesystem roots, numeric procfs processes, invalid runner shapes, and look-alike email domains.
  • Make host-path and email scanning linear-time and bounded against adversarial inputs.

Enhancements:

  • Add comprehensive corpus-based and mutation-based verification for every public allowance and refusal boundary.

Documentation:

  • Document the newly permitted public technical forms and their retained refusal boundaries.
  • Add detailed evidence covering corpus results, mutation proofs, fleet scans, and release verification.

Tests:

  • Add coverage for plain text, serialized tracker content, synthetic diffs, history changes, adversarial delimiter chains, long email inputs, Unicode path segments, and no-echo reporting.

Chores:

  • Update test fixtures and linked tracker metadata to distinguish reserved documentation addresses from personal addresses.

Summary by cubic

Fixes the import privacy gate refusing public technical text that names no person or machine, which stopped scheduled issue syncs (#193). The gate now allows narrowly scoped public forms — RFC 2606/6761 documentation email domains, the git SSH transport account, procfs/sysfs interface paths, GitHub-hosted runner workspaces, and placeholder home roots — while keeping concrete host data and personal email refusals intact.

Bug Fixes

  • Closes the hole where a refused path rode behind a permitted prefix ending in a delimiter: every non-name character now starts a new path judged by every host-path rule in one non-recursive pass, capped at 64 delimiter-started paths per match and failing closed beyond that. A property test confirms all 26 printable ASCII punctuation characters refuse; a fleet-wide comparison shows no added or lost host-path hits.
  • Judges env placeholders ($USER, ${USER}, %USERNAME%) in place so a closing brace or trailing percent no longer starts a second path, and gives the tilde rule the same boundary set so ], ), ; cannot publish a username.
  • Treats a bare closing tag naming one word (</var>, </root>, </data>, </tmp>) as markup, not a path start, so markup main allowed stays allowed; a closing tag that carries a path (</home/alice>) is still matched and refused.
  • Judges a Windows drive's colon as part of the path, not a delimiter, so a mixed-separator placeholder home (C:/Users/<user>\AppData) passes while a path smuggled behind a placeholder with mixed separators still refuses.
  • Extends embedded-identity refusals to tmp, private, Volumes, mnt, media, srv, and root, excluding a refused root that continues past /proc/self/root; only the bare procfs root entry after self is exempt, and nothing below it. Concrete numeric procfs pids are refused; /proc/ with a trailing slash stays allowed. The hosted runner's own tmp scratch inside its workspace stays allowed.
  • Rewrites the email-domain regex in linear time; the old pattern backtracked quadratically on long domains and stopped reporting empty labels (a fail-open gap). The rewrite accepts exactly the same language by fuzz comparison.
  • Judges path segment names by whole code points with combining marks as name characters, so a decomposed accent or astral letter no longer acts as a delimiter that hides a refused root path.
  • Compares Windows path tokens in their JSON-escaped form in the no-echo assertion, so a backslash path echoed into a gate report can actually fail the check.
  • Closes the noreply-identity follow-up by rewriting the pm-github#214 commits that used a mistyped noreply domain; their trees are unchanged.
  • Restores a numeric exit code in the proof main() test so its process.exitCode reset works under Bun and bun test exits 0.
  • A standalone host path with a doubled leading slash still evades matching on main; it is tracked as pm-github-8l0i rather than widened here.