SOW-0097: Single ownership of journal behavior in the SDK, used by both journalctl rewrites

KT
2d ago
+5273-2002

Pending user review

Agent-chosen implementation choices (recorded in SOW-0097):

  • Export/JSON: new EntryEncoder (Go NewEntryEncoder + AppendExport/AppendJSON/AppendJSONPretty; Rust EntryEncoder::new + append_*) and the public printable rule IsPrintableText / is_printable_text. Follows systemd v262 output_export/output_json exactly; JSON keys are sorted (systemd's order is hash order).
  • Boot listing: kept the algorithm both CLIs already used and the query matrix validates, now read straight from the _BOOT_ID DATA entry arrays (no match flush, no position change). Only canonical lowercase _BOOT_ID values name a boot, as the old CLIs effectively did. New Reader.ListBoots (Go) and FileReader::list_boots (Rust).

Open decisions (tracked in SOW-0176; current behavior unchanged):

  • 8: legacy ExportEntry / export_entry_bytes keep the SOW-0043 lossless contract instead of becoming systemd-exact.
  • 9: Rust DirectoryReader::list_boots / SdJournal::list_boots keep the infallible Vec signature (header-summary fallback on read errors); Go returns an error.

Summary

SOW-0097: one owner per journal behavior. Behaviors the journalctl rewrites copied from the SDK now live in public SDK functions in both Go and Rust, and both CLIs call them:

  • directory file listing and path dedupe, header flags, export seqnum ID, unit-name SipHash (Go), journald field-name rule;
  • export/JSON rendering and the printable rule: both CLIs now match the v262 goldens in all six cases where they previously differed;
  • --list-boots: the SDK listed one boot per file; it now lists every boot. The SDK and both CLIs handle dirty archives and corrupted files the way v262 does.

Also: a production duplication gate in CI with a committed baseline (76 pairs / 5167 lines, down from 79 / 5299), and the single-ownership rule in AGENTS.md.

Performance

  • Export/JSON, 100k entries: Go 5.18 s -> 4.56 s (export), 7.93 s -> 6.38 s (json), 9.19 s -> 6.78 s (json-pretty); Rust 2.01 s -> 1.65 s, 2.66 s -> 2.09 s, 2.63 s -> 2.11 s. Output is byte-identical before and after.
  • --list-boots, 200 boots x 1000 entries: Go 33 -> 30 ms, Rust 10.1 -> 9.2 ms (v262 19.6 ms). 100k entries without _BOOT_ID DATA: Go 3.61 s -> 77 ms, Rust 1.50 s -> 46 ms.

Validation

  • go test ./... and cargo test --workspace pass.
  • journalctl query matrix vs the v262 oracle: 3586 cases pass, follow included.
  • Directory matrix (34 checks), mixed-directory matrix (42/42) and verify matrix (63 results) pass.
  • Docs: wiki check passes, and all 39 verified examples pass, including four new ones.
  • Old-vs-new --list-boots sweep over 1409 corpus files and 288 directories: no changes except 3 corrupted fixtures, which now print the v262 result.

Stack

Based on #25 (SOW-0174). Merge after it.

Summary by cubic

Makes the SDK the single owner of journal behavior in Go and Rust: both journalctl rewrites now call public SDK functions instead of carrying their own copies, and export/JSON rendering and boot listing now match systemd v262 exactly.

New EntryEncoder (Go NewEntryEncoder, Rust EntryEncoder::new) renders export, JSON, and JSON-pretty per v262; both CLIs match the v262 goldens in all six cases where they previously differed. The printable rule it uses is public and also drives short, cat, and verbose output.

ListBoots/list_boots reports every boot from _BOOT_ID DATA entry arrays instead of one per file header, and handles dirty archives and corrupted files like v262; Go refreshes before reading postings so long-lived readers see new entries.

  • Directory file listing and path dedupe, header flags, long unit-name SipHash, the journald field-name rule, version-4 UUIDs, and disposed-file microsecond naming are public SDK functions both CLIs call; the Rust Netdata layer classifies non-UTF-8 names as Go does.
  • Legacy ExportEntry/export_entry_bytes keep the lossless contract, now length-prefixing unsafe metadata and skipping fields with no values; open contract decisions are tracked in SOW-0176.
  • Export/JSON get faster (Go export 5.18s → 4.56s per 100k entries; boot listing without _BOOT_ID DATA drops from 3.61s to 77ms), and a CI duplication gate pins production clones at a committed baseline (76 pairs / 5167 lines, down from 79 / 5299).