SOW-0161: source-timestamp index counts the captured entry population once
KT
1d ago+388+388+012345678901234567890123456789-40-40-01234567890123456789
Stacked on #18. Merge order: #13 → #14 → #15 → #16 → #17 → #18 → this. Completes SOW-0161 (.agents/sow/done/SOW-0161-20261017-file-indexer-source-posting-membership.md).
Decisions for review (agent-chosen, pending your review)
- Membership rule: the source-timestamp histogram uses the same rule the field-bitmap path already used.
- Postings past the captured tail are ignored.
- A posting the global entry array doesn't contain fails indexing (
MissingOffset) on a stable file. - On a live file (online, or active file name) such a posting is dropped.
- Live tolerance in the bitmap path: the field-bitmap path now also drops these postings on live files instead of failing.
- Reason: the Rust writer links an entry into the DATA arrays before it increments
n_entries. - In that window a live reader legitimately sees DATA postings for an entry the global array doesn't count yet.
- Reason: the Rust writer links an entry into the DATA arrays before it increments
- Repeated source field: an entry with the source field twice keeps its earliest timestamp and counts once.
- No
CACHE_VERSIONbump: cached indexes differ only for corrupt files or files with repeated source fields, and a bump would reindex every cached file.
Go has no equivalent surface: journal-index is Rust-only, and the Go explorer builds no posting-based histogram.
Reproduced before the fix (regular and compact)
- Rows published after the captured tail: counted (16 for 14).
- Source posting naming a non-ENTRY object: accepted (15 records for 14 entries).
- Repeated source value: double-counted (5 for 4).
- Link-before-count live window: extra entry counted.
Validation
journal-index/tests/source_postings.rs: 5 new tests. All fail on the old code and pass on the new.cargo test --workspace: 46 suites pass.- Cost: one comparison per posting and one counter per global entry. The member set is built only on a mismatch.
Summary by cubic
Fixes the source-timestamp histogram so it counts the captured ENTRY population once per entry, sharing the field-bitmap path's membership rule.
- Postings past the captured tail and postings naming no entry of the global array are dropped on every file, stable or live, in both the source and requested-field paths; live writers link DATA before counting an entry, and pre-v0.9.0 SDK writers could close a file with the links of a failed append.
- An entry with the source field more than once keeps its earliest timestamp and counts once.
CACHE_VERSIONbumps to 5, so every cached file reindexes once after upgrade.- Validation: 4 tests in
source_postings.rsreproduce the old behavior on live and stable files and pass on the fix;cargo test --workspacepasses 46 suites.