fix(e2e): honor cleanup grace when retiring workers

Merged
KN
6d ago
+100-6
  • Workers retiring after a run unit or draining at run end were killed after 10 seconds, even when cleanupTimeout allowed longer. A late disposal error could disappear and leave the run successful.
  • Use the existing forceGraceMs (cleanupTimeout plus the runner's 1-second margin) for both shutdown paths. Keep forced termination bounded.
  • Add scheduler regressions for retirement, final drain, late cleanup errors, and a shorter shutdown grace, plus an e2e patch changeset. This restores the cleanup budget already documented in the config and engine references.
  • Split from #1223 as requested in https://github.com/tester-army/e2e/pull/1223#issuecomment-6295762226. No eval code or authoring guidance changes.
flowchart LR
  A[Retire or drain worker] --> B[Dispose within cleanupTimeout]
  B --> C{Worker exits within forceGraceMs?}
  C -- yes --> D[Preserve cleanup errors]
  C -- no --> E[Kill worker]

Verified

Ran it locally: yes

  • Built the packages on main d7c6b5e5 and this branch, then ran the same standalone e2e run probe with a custom engine whose dispose waits 12 seconds. No model calls. The table compares final-drain behavior; scheduler unit tests separately cover retirement.
Probemainthis branch
12-second disposal, 30-second cleanup budgetHook killed before completion; exit 0Hook completes; exit 0
Same disposal, then throw controlled disposal failureError lost; exit 0ENGINE_FAILURE in cleanup; exit 3
12-second disposal, 7-second cleanup budgetCLEANUP_TIMEOUT; exit 3CLEANUP_TIMEOUT; exit 3
Four scheduler disposal regressions4 fail4 pass

The fixed CLI reports the late error:

ERROR  infrastructure error ENGINE_FAILURE (cleanup)
controlled disposal failure

Test Files  1 passed (1)
     Tests  1 passed (1)
    Errors  1 error
  • Deterministic Chromium validation used the built CLI through the testbed (79 passed, 1 skipped) and web benchmark (91 passed). Full repository checks and a fresh-context independent code review also passed. Native device behavior was not exercised locally.