feat(ENG-4364): opt-in tier3.dropShapeOnlyFields for whole-result reviews

Merged
HI
1d ago
+474-3

What

A new opt-in Tier 3 option, tier3.dropShapeOnlyFields (default false), for whole-result reviews (tier3_only and cascade with cascadeReview: "result").

With it on, the field: value serializer leaves a line out of the reviewer input when both hold:

  • the key is a plain field path of short field names, e.g. data.nodes[0].status.id: each key segment is at most 32 characters and at most 3 words (split on _, -, $, @ and camelCase);
  • the whole value, trimmed, is one of:
    • null, undefined, true or false;
    • a number;
    • a UUID or a hex ID (8–64 hex characters, at least one digit);
    • an ISO date or date-time;
    • a hex colour.

These values can't carry an instruction.

Why

List-heavy tool results serialize mostly to lines like items[0].id: <uuid>, items[0].canceledAt: null or items[0].progress: 0. They fill the review window (maxChunks Ɨ maxTextLength), so free text past it goes unreviewed. They also make each judge call longer and slower. Leaving them out puts the window on text.

Design choices

  • Where: the filter runs inside the serializer, before the cap and chunking, so the freed budget goes to text lines in the same pass.
  • Never left out:
    • lines of a multi-line value, so a bare number or date inside text stays;
    • keys with spaces or other characters (keys are attacker-controlled text);
    • empty values (key:);
    • top-level values.
  • Paths: matched with ASCII word characters only, so non-ASCII keys always stay.
  • URLs and short words stay. Dropping them also removed pieces of injected multi-line values in testing.
  • Coverage accounting: a left-out string counts as covered, not as dropped for budget. Filtering alone never sets oversize or coverageDegraded. A result with nothing else left skips the provider, as an empty result does today.
  • Scope: only the reviewer's input changes. Tier 1, Tier 2, SFE and the returned result are untouched.

Tests

  • specs/tier3-shape-filter.spec.ts:
    • shape-only values left out; text, URLs, emails, short words and empty values kept;
    • non-plain keys, text-like keys and multi-line values kept;
    • hex-looking words without a digit kept;
    • off by default;
    • coverage: left-out values don't flag oversize, an overflowing result is fully covered with the filter on, and real overflow is still reported;
    • the provider is skipped when nothing is left.
  • Five new tier3-serialize conformance cases for the Python port, and the rule in conformance/README.md.

pnpm test, pnpm lint and pnpm test:typecheck pass.

šŸ¤– Generated with Claude Code