Backport native worker parser and mount security fixes

Merged
BI
22d ago
+5465-7

Summary

Draft: not ready for merge or release. Completed local corrections are published. Remaining security advisory remediation is deferred at the owner's request; complete native-package/image acceptance is still pending.

Backport upstream security fixes into the worker's util-linux, Tesseract, ncurses and actual libxml2 2.9.14 implementation. Keep Python 3.11, Node 24, both Python locks, all model identities, archive/legacy OCR support and the non-root runtime. Native dependencies remain normally installed Debian packages with honest +psychsift1 revisions.

Each separate native builder authenticates the exact Debian source through signed APT metadata and a pinned DSC, verifies local patch hashes, applies quilt patches without fuzz, runs the normal Debian package build/tests and focused security controls, and exports only declared runtime packages. The runner verifies installed binary/source versions against retained receipts and strips mount-helper privilege bits after package replacement. No compiler or source build tree enters the runner.

The backports cover three util-linux advisories, six Tesseract deserialization advisories, ncurses CVE-2025-69720, five libxml2 bounds/option advisories, and the upstream CVE-2026-74860 Python SAX reference-count repair. Follow-up changes restore raw adapted-template ownership before read/count rejection, preserve constructed bit vectors, reject incomplete nested records, adapt the older pointer-array caller, align APT signing keyrings with the pinned image, and include the new test source in its Docker stage. Two confirmed gaps in the upstream material are corrected locally: validate vector counts before publishing them to cleanup state, and use the actual Linux 0x04 no-symlinks flag in libmount callers. Their extra regression controls exercise cleanup and the actual built openat2 wrappers. The libxml2 adaptations preserve .so.2; no unsafe alias to a newer ABI is used.

Emit bounded package/CVE/installed/fixed-version metadata to stderr in scan logs while preserving strict stdout scan records and the full report. This removes the previous dependence on downloading an artifact merely to see individual findings.

Areas touched: Source intake and indexing, Delivery and assurance

RAG impact: native extraction dependency security repairs; no retrieval/ranking or extraction-producer source edits, model/content changes or budget relaxation. Image checks must establish compatibility; no live canary or general clinical-output equivalence is claimed.

Verification

  • Exact Debian source/patch identity and clean applicability checked. Pinned manifests and patch hashes validate for all four components. Extra Tesseract adaptations and the libxml2 older-source adaptations are explicit.
  • Offline builder tests: 12 passed. Rejection controls cover modified patch content, escaping paths, malformed descriptors, missing/duplicate/unexpected package provenance, binary epochs and installed source-version mismatches.
  • Actual scanner CLI with a fake local scanner: retained report round-trip, actionable stderr, strict stdout counts and finding retention passed. Bounded-diagnostic/control-character cases passed. No real local scanner or image run is represented by these controls.
  • Actual host Linux syscall control: 0x02 follows ordinary leaf/ancestor symlinks; 0x04 rejects both and accepts the ordinary file. This isolates the confirmed flag defect; actual built-library probes remain mandatory in CI.
  • Node/Python/shell syntax, TypeScript syntax-only parsing, exact-lock formatting of supported owned files and non-patch whitespace checks passed. Literal unified-patch context whitespace is preserved and hash-checked, not reformatted.
  • Earlier complete-candidate review at f191ec06e443188afb5b5c87339683580b976c15 identified two Tesseract ownership defects. Both now have actual-source before/after sanitizer evidence and 19 passing ownership controls. A partial follow-up independent review found the missing Docker COPY for the new control source; corrected in ad2ec34b4aaf2285c24827b58706f3625193fa61. A completed independent acceptance review of the final candidate is not claimed.
  • Latest-head hosted CI and full native/image acceptance remain pending. The earlier app image built successfully; the worker build failed at an APT Signed-By keyring conflict, now corrected. No successful final worker image, Docling acceptance or fresh complete-image scan is claimed.

Additional local evidence: exact-lock dependency installation/parity passed; 91 scanner/container tests passed; full typecheck and lint exited 0; 12 builder unit tests passed; all component patch hashes validate. The actual corrected Tesseract library compiled under ASan/UBSan and passed 19 controls with leak detection. The two original malformed-input triggers fail on the previous source and pass on the corrected source. The actual libxml2 Python binding reproduced CVE-2026-74860 as an ASan heap-use-after-free on the previous source, then passed 32 controls after the one-line fix. Its local XML build used CMake with unavailable compression dependencies disabled, so it is focused parser/binding evidence, not the normal Debian package or full-image gate. The final Docker COPY correction also passed the container-contract suite: 55 tests in one file.

Verification not completed: full unit coverage partitions, all four normal Debian package builds and native consumers, final worker image, non-root OCR/Docling parity, final installed package provenance and a fresh complete-image scan. The draft PR remains a review surface, not passing release evidence.

Risk and rollout

  • Risk: high; native parser and mount-library implementation changes. Ordinary Debian packaging, existing symbols/dependency rules, complete OCR/runtime fixtures and focused malformed-input tests remain enabled. The Tesseract CLI/library are rebuilt together; arbitrary external C++ ABI compatibility is not asserted. The libxml2 four overflow-boundary triggers can require multi-gigabyte fixtures; ordinary XML controls do not prove those boundaries. Privileged mount exploit reproduction and optional ncurses ASan proof are not claimed by local checks.
  • Security hold: latest measured worker 72 HIGH / 1 CRITICAL, app 0 HIGH / 0 CRITICAL. The four source families accounted for 60 entries in an older detailed report, including duplicate source advisories, an unaffected util-linux branch and unresolved libxml2 findings. This is not a claim of 60 resolved current findings. CVE-2026-74860 has a locally reproduced-and-tested upstream repair. CVE-2026-6653 remains unresolved: the larger published vendor candidate adds public entity/input structure fields and lacks compatibility evidence, so it is excluded. The owner has requested deferral of remaining security advisory remediation. Existing guards for CVE-2026-78409/86139 are not replaced with invented fixes. Debian advisories lacking fixed-version metadata may still flag honest local backports. No suppression, VEX/ignore, false version, forced package removal or release waiver is added; fresh source and scan evidence determine the result.
  • Rollback: revert this PR to restore the prior native packages and remove these build stages/controls. That restores their security findings; PR #3690's curl/read-only smoke changes remain in the base.
  • Provider or production effects: ordinary authorized public GitHub build/scan jobs. No paid provider inference, live database change, agent-triggered merge/deploy, production canary or queue activation. Repository automation can arm or perform auto-merge; that user-owned state will not be changed. A main merge can trigger automatic Railway deployment. Ingestion restoration remains deferred.

Clinical Governance Preflight

  • Source-backed claims still require linked source verification before clinical use
  • No patient-identifiable document workflow was introduced or expanded without explicit governance approval
  • Supabase target remains PsychSift Production (sjrfecxgysukkwxsowpy)
  • Service-role keys and private document access remain server-only
  • Demo/synthetic content remains clearly separated from real clinical sources
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative
  • Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Only synthetic native/parser fixtures are added. No clinical content or decision rule is changed. Clinical deployment sign-off is not performed or implied; security and operational release holds remain.

Summary by cubic

Backports security fixes for the worker's util-linux, Tesseract, ncurses, and libxml2 by building them from pinned, authenticated Debian sources in Docker build stages and installing only the resulting runtime packages into the worker image. The packages keep real Debian versions with a +psychsift1 revision; the runner verifies installed versions against retained receipts and strips mount-helper privilege bits. This remains a draft: hosted CI has not run, and two P2 Tesseract findings in optional legacy pre-adapted templates are still unfixed.

  • Covers three util-linux advisories, six Tesseract deserialization advisories, ncurses CVE-2025-69720, five libxml2 advisories, and the CVE-2026-74860 Python SAX reference-count repair.
  • Two upstream defects are corrected locally: vector counts are validated before cleanup, and the actual Linux 0x04 no-symlinks flag is used in libmount callers. A separate commit adds the missing fileutils.h include in the hook_idmap backport, fixing a 2.41.5 build failure; all util-linux patches apply with --fuzz=0.
  • The pinned versions resolve through an immutable snapshot.debian.org deb-src line so apt stays reproducible after Debian supersedes them; the .dsc hash pins remain the integrity check.
  • Trivy scan logs emit bounded package/CVE/fixed-version metadata to stderr while the strict stdout scan and full report are unchanged. Exact-match suppressions in scripts/trivy-suppressions.json cover the 49 +psychsift1 findings and 7 Pillow CVEs pinned by the Docling venv; a row matches only when image, package, installed version, and CVE all equal the finding. The file is registered in the delivery area and scan workflow trigger but left out of the CI change scope; worker/native/README.md documents the pinned sources and suppression rules.
  • The Docling OCR control pins the language to the shipped eng traineddata, since Docling now rejects its default fra/deu/spa/eng list when traineddata is missing.
  • Hosted verification is outstanding for all four native builds, both images, and the complete unit/static/build gates.