Implements the additive Task 6 publishing infrastructure from docs/plans/2026-11-05-flakiness-testing-dashboard.md, updated for current main's parallel unit job, four non-locked E2E shards, serial locked job, and local fork/Dependabot path.
Producer jobs preserve native Flakiness reports under suite/job/shard/run/attempt identities, including failed-test evidence, without new Flakiness credentials or OIDC permissions.
A separate workflow_run consumer checks out main only, installs the locked independent runtime with lifecycle scripts disabled, authenticates exact-attempt producer metadata from GitHub, and validates one complete artifact batch before any credential-bearing CLI process starts.
Immutable pre-checkout authorization-job metadata supplies historical PR repository/SHA even when fork pull_requests associations are empty. Live PR data is comparison-only; stale/missing identity fails closed.
Actual flakiness upload uses an absolute executable, fixed endpoint, configured project and isolated environment. CLI receipts remain accepted-unverified, not dashboard read-back proof.
Delivery records survive upload-only consumer reruns. Missing/expired/unsealed history and possibly-submitted states block blind resubmission. Proven pre-spawn failures can safely retry identical content.
Credential-bearing CLI HOME/TMPDIR are outside the durable ledger. An always-run flat atomic exporter publishes only checkpoint/hash-named regular records, including after abrupt termination; no credential caches are archived.
Activation and limits
Publishing defaults off. The consumer also requires vars.FLAKINESS_UPLOAD_ENABLED == 'true'; that variable is currently absent (repository variable-name readback verified). This PR does not change repository variables, service/project settings, secrets, cutover, or legacy stats reporting.
The consumer and revised producer metadata must first exist on trusted main. pull_request_target uses the base workflow, so this head's hosted PR Tests run exercises base CI against the exact head, not the new producer artifact steps. Local integration tests exercise the new producer-layout/consumer path with actual pinned native reports and fake no-network CLI/API boundaries.
Task 7 privacy/project approval, representative hosted failure/size/count/attachment parity, OIDC delivery and exact dashboard read-back are required before separately approving activation. All-or-nothing completeness and fixed intake budgets remain deliberate; incomplete/mismatched evidence is withheld visibly while native artifacts remain. Partial producer 're-run failed jobs' attempts are unsupported; use a complete fresh producer attempt.
Details and recovery contract: docs/testing/flakiness-publishing.md.
Verification
At 737d3f82aa341c15dece30bed5fdc637fc4aeed5:
Local reporting: 398 passed, with network denied; includes actual native Vitest/Playwright report/attachment-to-validator-to-fake-CLI execution, abrupt SIGKILL credential isolation, ledger export/restore, fork/stale identity, shard completeness and workflow-name mutation regressions.
Linux-like unset-TMPDIR subset: 48 passed. Preservation comparisons use reachable merged-main history, not unpublished local commits.
npm run type-check (core tsc --noEmit), actionlint on all three relevant workflows, and git diff --check: exit 0.
Fresh independent pinned-runtime npm ci --ignore-scripts and actual flakiness upload --help: exit 0. This is not hosted auth/delivery proof.
Exact-head hosted CI, including E2E: in progress; no green claim yet.
Original 18 inline findings and the two later CI-environment findings have individual fix replies and resolved-thread readback; final bot inventory and final-head independent Opus review are still being verified.
No merge is performed by this workstream; parent retains the separate all-checks-green and after-20:00-Pacific merge gate.
Summary by cubic
Implements Task 6 of the flakiness dashboard: PR Tests uploads native Flakiness reports as artifacts, and a new isolated consumer workflow validates them and publishes to flakiness.io/SonicJs/sonicjs.
Producer and consumer
pr-tests.yml uploads flakiness-reports/ from unit-tests, the four E2E shards, serial locked, and local jobs, keyed by suite, job, and run ID/attempt, with if: always() so failed-test evidence still reaches the dashboard.
flakiness-upload.yml runs only on completed PR Tests runs from the pinned workflow and accepts original push and pull_request_target events; it checks out main only (never PR code), validates artifacts against GitHub run/jobs metadata, authenticates via GitHub OIDC, and posts receipt URLs to the job summary, gated by the FLAKINESS_UPLOAD_ENABLED variable.
Producer identity comes from an immutable authorization job name; older runs without it can't be published, so all producer jobs must be rerun after rollout.
Delivery safety
Consumer reruns require the previous attempt's delivery ledger checkpoint and fail closed otherwise, so interrupted or terminated consumer attempts never resubmit; a YAML contract test keeps the restore-failure gate that blocks the upload helper.
Recovery history is sealed: only single-link checkpoint and hash-named records are exported, rejecting unsafe entries and never archiving credential caches.
Adds provenance fetch, delivery ledger, and upload entrypoint scripts plus integration and YAML contract tests; no new credentials or id-token: write were added to test jobs, and fixtures now write to a gitignored scratch root.
The consumer workflow only starts firing after this PR merges to main, so the first upload happens on the next PR Tests run.
Confidence Score: 5/5
The PR appears safe to merge with publishing still disabled and the separate activation gates unchanged.
Summary
Adds a separate, default-off workflow that publishes validated Flakiness reports without giving test jobs upload credentials.
PR Tests saves reports from each test path and keeps a complete required check.
A gated workflow checks a complete test run before publishing its reports.
Upload retries keep a sealed record of what may already have been sent.
A push-only workflow checks the full local test path on the exact head.
Reporting tests keep their scratch files outside producer report artifacts.
The three supplied, unnumbered previous threads are addressed: suite folders survive upload, reports are validated together, and the CLI executable uses an absolute path. No new actionable issues were found. Hosted OIDC delivery and dashboard read-back remain separate activation gates; this review did not run them.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[PR Tests completes] --> B[Preserve native reports]
A --> C{Publishing enabled and event allowed?}
C -->|No| D[Skip consumer job]
C -->|Yes| E[Load main-only helpers]
E --> F[Check GitHub identity and complete report batch]
F -->|Rejected| G[Fail without upload]
F -->|Accepted| H[Check saved delivery records]
H -->|Unsafe retry| G
H -->|Safe to submit| I[Run isolated upload CLI]
I --> J[Record accepted-unverified or failure]
G --> K[Export safe delivery records]
J --> K
