fix(security): bound gzip reads and coverage spans (#228, #229)
DO
2d ago+562+562+012345678901234567890123456789-49-49-01234567890123456789
Summary
Fixes two low-severity resource-exhaustion findings from the CodeRabbit security scan. Both were reproduced before fixing.
#228: decompression bomb in gzip readers
- Confirmed:
_iter_ndjsonandLocalStoreiteratefor line in fh(and calljson.loadon whole raw windows) on a gzip text stream, so one line or document is fully expanded in memory before any check runs. - New
bm/safe_io.py:iter_bounded_lines()readsreadline(limit + 1)from the binary (de)compressed stream, so it never buffers more than the capload_json_document()reads at mostlimit + 1decompressed bytes
- Used by
scan_device_health_from_file(NDJSON and CSV), and byLocalStoreraw readings/notes windows,readings.ndjson(.gz)andnotes.ndjson. - Going over a limit raises
InputLimitError(aValueError, exported frombm). The message names the file, the limit and the env var that raises it. The battery CLI prints it and exits 2. - Corrupt or truncated gzip windows are still skipped as before. An oversized window is not skipped silently: the read fails so the operator sees it.
#229: unbounded 180-day window list
- Confirmed:
LocalStore._missing_windows([], now - 10**9 days, now)built 5,555,556 windows in 5.3 s at 513 MiB peak. The input is reachable from the MCPdays/stale_days/lookback_daysarguments, and larger values scale linearly toward OOM. ensure_coveragenow raisesValueErrorfor spans overBROODMINDER_MAX_COVERAGE_DAYS, before any manifest or network work.- The offline staleness check and the post-sync check now use a new
_gaps()helper. It returns at most one gap per completed interval, plus one, and does not enumerate windows._missing_windowsis only used on the auto/fetch path, where the span is now bounded.
Limits chosen
| Env var | Default | Rationale |
|---|---|---|
BROODMINDER_MAX_LINE_BYTES | 1 MiB | A reading row is under 2 KiB |
BROODMINDER_MAX_DOCUMENT_BYTES | 128 MiB | A 180-day raw window is ~5 MB per device; leaves room for large hives |
BROODMINDER_MAX_COVERAGE_DAYS | 7305 (20 years) | Longer than BroodMinder history; ~41 windows per hive |
An invalid value (non-integer or ≤ 0) fails with a clear ValueError. Limits are documented in the README Configuration section and in the CHANGELOG under Unreleased → Security.
Not changed (out of scope): scripts/flatten.py reads gzip windows it wrote itself. The streaming total size is not capped, because memory use is bounded per line and a large legitimate archive must still stream.
Test plan
- New regression tests in
tests/test_battery.py(5) andtests/test_store.py(9). They cover over-long lines (gzip NDJSON, gzip CSV, flat archive,notes.ndjson), oversized raw readings and notes documents, the CLI exit code, invalid env values, rejected unbounded spans (offline and auto, with no client calls), a configurable span limit, an offline gap check that does not calliter_windows, and MCPtelemetry_trends/hive_noteswithdays=10**9. - Full suite:
343 passed, 6 skipped(skips are live API and GitHub-token checks). -
python -m compileall bm scripts tests
🤖 Generated with Claude Code