fix(web): expose native file choosers to agent uploads

Closed
PU
20d ago
+669-56

What and why

  • Clicking a visible upload button can open a native picker for a hidden file input, leaving the agent without an upload target. Expose that pending picker as a fresh observation reference and reuse the existing upload action.
  • Keep project-relative file authorization, masking and single/multiple-file checks. Navigation, replacement, cancellation, submission and attempt teardown retire the reference; Escape preserves an existing FileList.
  • Add native Chromium and protocol regressions, update the reference/skills and give existing fake Page/Context fixtures native event semantics. No public action schema or dependency changes.
flowchart LR
  A[Click visible upload button] --> B[Native picker event]
  B --> C[Fresh observation reference]
  C --> D[Existing upload path policy]
  D --> E[Original picker setFiles]
  E --> F[Retire reference]

Issue

Fixes #954

Verified

Ran it locally: yes

mainthis branch
Test-first native hidden-input regression fails because no chooser target is observed.Native chooser regression passes with exact uploaded bytes.
Published core/web pair: the real agent clicks the visible upload button but cannot attach the four approved files.Paired packed core/web candidate passes once through the real agent: one multipart upload, four independently checked byte counts/SHA-256 values and exact visible success.

The actual paired-package loopback ran with direct OpenAI gpt-6-luna/high, no retries. Run 01a175f0-c15e-7690-85b5-4e595afdbf3e: 1/1 passed, 32.186s, four model calls, cleanup complete. Original failure retained. Only local test files were uploaded to loopback; this is not production application import acceptance.

✓ real agent uploads four approved PDFs through a hidden native multi-file input 31.61s
Test Files  1 passed (1)
Tests       1 passed (1)
Duration    32.19s

The native regression is reproducible with pnpm --filter @e2e-dev/web exec vitest run tests/integration/file-chooser.test.ts after building core/web.

pnpm check passes. Core tests pass 3,039 unit + 781 integration, with one existing unit skip. Web tests pass 605 with one existing skip. The built CLI testbed passes 79 with one existing skip; the deterministic web benchmark passes 91. Control-inventory strict replay passes 7/7 with zero model calls/tokens. Focused native/policy coverage includes stale references, supersession, Escape preserving files, owning-frame navigation, directory/multiple constraints and masking.

Current core 0.21.3 and web 0.15.3 were built, packed together and verified byte-for-byte after installation. Compatibility with the oldest declared core peer was not qualified, nor were the complete native/mobile/release matrix. A bounded two-pair bench:ab --suite testbed --min-pairs 2 --max-pairs 2 --warmup 0 --no-build -- tests/controls.e2e.ts comparison also completed with both builds exiting zero in every pair. No performance claim is made. Private fixture documents, model traces and reports are not attached. gh 2.96 lacks --attach; CLI evidence is included above.