fix(budget-breaker): lengthen the early exit's default slot from two minutes to ten

Merged
DO
1mo ago
+44-15

Problem

"feat(budget-breaker): wake on agent workflow runs" (#2293) estimated about 47 breaker runs an hour. The first 84 minutes of real runs showed 309 created and 115 executed (about 82 an hour), with no self-trigger loop: fleet activity is much burstier than the daily average the estimate used. With a two-minute early-exit slot, that is up to 30 organisation scans an hour on the maintainer login's shared API quota, for a breaker that is disarmed and decided none every time. Measurements are on #2259.

Design

One default changes: BUDGET_BREAKER_MIN_INTERVAL_MIN goes from 2 to 10 minutes when the repository variable is unset. Nothing else about the early exit changes.

  • At most 6 organisation scans an hour instead of 30. The two cheap calls of a run that exits early are unchanged.
  • A quota marker posted in this repository still makes the next run decide at once (the early exit's own check).
  • A marker posted only in another repository is read by the first run of the next slot: up to ten minutes later instead of two.
  • The number of runs created does not change. Only a different trigger can change that; filed as #2306.

The maintainer chose this over leaving it for a day or reverting #2293.

Risk

  • A trip driven only by other repositories' markers can be up to eight minutes slower than before. The decision still looks at a 30-minute window, so evidence is not lost.
  • During a real quota event with markers in this repository, every run in the slot scans (the early exit does not apply). That was already true; a longer slot does not make it worse.
  • The breaker is disarmed and BUDGET_BREAKER_ARMED is unset.

Test plan

  • tests/dev-lead/unit/test_budget_breaker_writer.bats: 97 of 97. The default-slot assertions now expect ten minutes; the tunable test sets two explicitly, and gains two cases for the default: a wake-up 590 seconds into the slot exits early on a run created at its start, and one 610 seconds in (the next slot) decides.
  • shellcheck --severity=warning -x clean.

Rollback

Revert, or set the repository variable BUDGET_BREAKER_MIN_INTERVAL_MIN to 2.

Monitoring

Breaker run logs: the share of runs that print early exit should rise, and quota markers read should appear at most six times an hour outside a quota event.

Refs #2259

šŸ¤– Generated with Claude Code