fix(classifier): harden rule matching, GitHub sync and Drive ingester scope

DO
3d ago
+2074-1088

Summary

Phase 0 of a categorization architecture review: bugs that were silently misfiling mail, risking note data, or burning Gemini quota.

Rule engine (gmail-ai-classifier)

  • Word-bounded subjects in any-mode rules. containsAny_ is a substring match, so the UniFi rule (unifi, subject alone from any sender) archived any subject containing "unified" into Tech/Alerts at 0.99. New containsAnyWord_ enforces boundaries on alphanumeric term edges only.
  • Field-scoped exclusions. The first commit on this branch checked every exclusion against the body too; words like bill, due, security, login, invoice appear in routine PayPal receipts ("Billing address", "Invoice ID"), newsletters ("national security") and changelogs ("SSO login"), knocking them off the fast path. Body exclusions are now word-bounded and skip a short list of subject-only words.
  • EOB fast path had no exclusions; denials, records requests and balance-due notices are now excluded.
  • Career shield: generic phrases (introductory call, opportunities at , discuss the opportunity) now require recruiting context, so cold sales and volunteer mail are no longer pinned to Work/Career in Primary.
  • Signature shield: (isDocuSignSender && isDocuSignRequest) || isDocuSignRequest reduced to the second term; requests are now filed by content (school / medical / employment / Finance/Legal) under canonical labels instead of the non-canonical Legal/Legal_Court.
  • Security shield: card/account fraud alerts from financial senders file under Finance/Banking (still Primary).
  • School: tuition bills are no longer treated as broadcasts. Domain-list entries containing @ now match that exact sender.
  • Recovery loop: revertAndQueueMissedCareerEmails ran on every 5-minute trigger and re-queued correctly-filed job-board promos titled "Job Offer" each cycle (clearing their retry count each time) for 14 days. It now runs only from runMissedEmailsSweep, and its query excludes promotions/social/forums.

GitHub sync

  • code.gs and gitHubSync.gs both defined 7 GitHub helpers; Apps Script shares one global scope, so gitHubSync.gs silently won at runtime while Jest tested the code.gs copy. The duplicates are removed; code.gs/src/index.js require gitHubSync.gs under Node only.
  • Files over 1 MB come back from the Contents API with encoding: "none" and empty content; the sync decoded "" and would have PUT a one-entry note over the whole file. It now refuses (FILE_TOO_LARGE, not retried).
  • Idempotency keys on an invisible <!-- source: gmail thread=… msg=… --> marker instead of generated prose (which duplicated on rewording and dropped same-day lookalike receipts).
  • Optional NOTE_LOG_SHARDING (quarterly/monthly) writes to <domain>/activity/<period>.md. Default unchanged.
  • Missing repo config is non-retryable; last outcome recorded in GITHUB_SYNC_LAST_STATUS.

Drive ingester

  • Query limited to 'me' in owners, and only files under a canonical domain folder are ingested (it previously scanned every Doc visible to the account, including docs shared by others).
  • Tags go in the Drive description only; Google Doc bodies are no longer modified with a YAML block.
  • Retired gemini-2.0-flash / gemini-1.5-flash endpoints replaced by gemini-2.5-flash-lite; marker idempotency; sharding.

Tooling

  • Jest ignores .claude/ worktrees (every suite ran once per worktree).

Testing

  • New tests/phase0-hardening.test.js (both code.gs and src/index.js) plus updated drive-ingester tests.
  • npx jest --runInBand --coverage: 25 suites, 1220 tests passing; coverage 97.8 / 87.09 / 98.98 / 99.05 (thresholds met).
  • npm run check (prettier, eslint, PII) passes; code.gs ↔ src/index.js parity verified.

Summary by CodeRabbit

  • New Features
    • Drive and email activity can be organized into monthly or quarterly notes.
    • Email filing better recognizes payment receipts, signature documents, career-related messages, and financial security alerts.
    • Email note creation supports excluding selected sub-labels while retaining Gmail filing.
    • Drive summaries include a source reference, and Drive tagging appears in file descriptions.
  • Bug Fixes
    • Drive ingestion is limited to owned, non-trashed documents in recognized folder trees.
    • Duplicate email and Drive entries are less likely to be added again; existing GitHub notes are protected when their content cannot be read safely.
    • Email recovery searches skip Promotions, Social, and Forums; classification matching is more precise.