feat(connectors): draft check runs can run, re-run, skip or stop one check

Merged
EV
19h ago
+1002-59

Description

Backend PR for the per-check row actions of the connector checks card (Re-run Check, Run Check, Skip, Start Now). Stacked on #16207. The frontend (C4) uses this API. No frontend change: the API is opt-in, and the form does not send the new fields yet.

Request

  • POST /admin/connector-checks/runs (draft runs) and POST /admin/cc-pair/{id}/edit/checks (pair dry runs) take three new fields (DraftCheckActions):
    • check_ids: only these checks run, and they ignore their cached results. Every other check keeps its cached result, or its result in the latest run of the same draft key. A check with no kept result runs. Example: after Run Check on a stopped check, its "Not run" dependents run too.
    • skip_check_ids: these checks do not run.
    • start_check_id (Start Now): this check, and the checks it needs, run first. A discovery check still runs after the validation checks.
  • These requests give 400 INVALID_INPUT (OnyxError): an unknown check id, a check in both check_ids and skip_check_ids, a skipped start_check_id, an empty check_ids, and check_ids with a rerun mode. The API rejects them before it writes anything.

States

  • New status stopped (CapabilityCheckStatus and DraftCheckStateKind) and new reasons skipped_by_user and stopped_by_user. The read-time enums did not accept stopped before; this PR adds it.
  • A skipped optional check is skipped (skipped_by_user, line "Skipped by you"). It never blocks.
  • A skipped required check is stopped (stopped_by_user, line "Not run: stopped by user"). It is no pass: the capability verdict is skipped, not passed. It holds Create.
  • Dependents of a skipped or stopped check are "Not run" with dependency_skipped and blocked_by, as for a check that found nothing to test. The lines are "Not run: {name} skipped" and "Not run: {name} stopped". A chain names the skipped check.

Creation

  • The API caches the skip of an optional check under the check's draft result key. Creation (and edit apply) on the same form reuses it, so the check does not run, and the stored report shows it as skipped by the user.
  • A stopped required check is never cached. Creation always runs it, and it blocks if it fails. A cached skip of a check that is now required is also not reused.
  • A later run that does not skip the check ignores the cached skip and runs the check.

Not in this PR

  • Stored (saved-credential) runs do not take the actions. The detail page does not need them yet.
  • Skip and Start Now on a running run start a new run that supersedes the old one, as a form edit does. The check that was running can run again in the new run.

How Has This Been Tested?

  • New unit tests (test_draft_check_actions.py). They call start_draft_capability_check_run and the draft run task in process, on a fake cache, without a server:
    • the stopped and skipped states, their lines, and their dependents
    • the cached optional skip, and that a later run ignores it
    • Re-run Check, and the results a run keeps from the cache and from the latest run
    • Run Check on a stopped check, and that its dependents run
    • Start Now order, also after a skip
    • every invalid request (400, nothing written, nothing sent)
    • the verdict for stopped and skipped_by_user
    • creation reuses an optional skip and runs a required one
  • Connector and server unit tests: 6838 passed, 25 skipped, 1 failed. The failure (test_browser_lifecycle.py) resolves example.com, and this session has no DNS. It does not use the changed code. EE unit tests: 1184 passed. ty and pre-commit are clean.
  • The integration tests for the API are not written. This session has no running Onyx stack, Postgres or Redis, so I could not run them.

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.
  • [Optional] Override Linear Check

Generated by Claude Code

Summary by cubic

Draft runs and pair dry runs get per-check actions: check_ids (Run Check / Re-run Check), skip_check_ids (Skip), and start_check_id (Start Now). The API is opt-in and the form doesn't send the new fields yet, so no frontend change is needed.

Behavior

  • check_ids runs only those checks, ignoring their cached results; other checks keep cached or latest-run results and run only when they have none.
  • skip_check_ids skips: optional checks become skipped ("Skipped by you"), required checks become stopped ("Not run: stopped by user") and hold Create; dependents show "Not run: … skipped/stopped".
  • start_check_id runs that check and everything it needs first, ahead of other checks of its kind; execution and progress share this order.
  • Invalid actions – unknown check_ids, empty check_ids, overlapping run/skip, skipping the start check, or check_ids with rerun – return INVALID_INPUT before any run starts.

States and caching

  • New stopped state and stopped_by_user/skipped_by_user reasons; a stopped required check is no pass (capability verdict skipped).
  • Skips are cached per user, apart from the shared results: creation reuses only the requesting user's optional skip, a stop hides the check's cached result so creation always runs it, and other users run the check. A later run that doesn't skip the check clears the skip.
  • A run saves its skips under the start lock after it becomes the latest, so a result of an older, superseded run doesn't overwrite a newer run's skip.

Summary by CodeRabbit

  • New Features
    • Capability-check runs now support selecting checks to rerun, skipping optional checks, and choosing which check to start first.
    • Required checks that are skipped are marked as stopped and can block connector creation; dependent checks are clearly marked as not run.
    • Check results and skip choices are preserved per user, while required checks can be rerun when needed.
    • Invalid or conflicting check selections now receive clear validation errors.
  • Bug Fixes
    • Improved ordering and reporting for checks with dependencies and cached results.