fix(classifier): harden rule matching, GitHub sync and Drive ingester scope
DO
3d ago+2074+2074+0123456789012345678901234567890123456789-1088-1088-0123456789012345678901234567890123456789
Summary
Phase 0 of a categorization architecture review: bugs that were silently misfiling mail, risking note data, or burning Gemini quota.
Rule engine (gmail-ai-classifier)
- Word-bounded subjects in
any-mode rules.containsAny_is a substring match, so the UniFi rule (unifi, subject alone from any sender) archived any subject containing "unified" into Tech/Alerts at 0.99. NewcontainsAnyWord_enforces boundaries on alphanumeric term edges only. - Field-scoped exclusions. The first commit on this branch checked every exclusion against the body too; words like
bill,due,security,login,invoiceappear in routine PayPal receipts ("Billing address", "Invoice ID"), newsletters ("national security") and changelogs ("SSO login"), knocking them off the fast path. Body exclusions are now word-bounded and skip a short list of subject-only words. - EOB fast path had no exclusions; denials, records requests and balance-due notices are now excluded.
- Career shield: generic phrases (
introductory call,opportunities at,discuss the opportunity) now require recruiting context, so cold sales and volunteer mail are no longer pinned to Work/Career in Primary. - Signature shield:
(isDocuSignSender && isDocuSignRequest) || isDocuSignRequestreduced to the second term; requests are now filed by content (school / medical / employment / Finance/Legal) under canonical labels instead of the non-canonicalLegal/Legal_Court. - Security shield: card/account fraud alerts from financial senders file under Finance/Banking (still Primary).
- School: tuition bills are no longer treated as broadcasts. Domain-list entries containing
@now match that exact sender. - Recovery loop:
revertAndQueueMissedCareerEmailsran on every 5-minute trigger and re-queued correctly-filed job-board promos titled "Job Offer" each cycle (clearing their retry count each time) for 14 days. It now runs only fromrunMissedEmailsSweep, and its query excludes promotions/social/forums.
GitHub sync
code.gsandgitHubSync.gsboth defined 7 GitHub helpers; Apps Script shares one global scope, sogitHubSync.gssilently won at runtime while Jest tested thecode.gscopy. The duplicates are removed;code.gs/src/index.jsrequiregitHubSync.gsunder Node only.- Files over 1 MB come back from the Contents API with
encoding: "none"and empty content; the sync decoded""and would have PUT a one-entry note over the whole file. It now refuses (FILE_TOO_LARGE, not retried). - Idempotency keys on an invisible
<!-- source: gmail thread=… msg=… -->marker instead of generated prose (which duplicated on rewording and dropped same-day lookalike receipts). - Optional
NOTE_LOG_SHARDING(quarterly/monthly) writes to<domain>/activity/<period>.md. Default unchanged. - Missing repo config is non-retryable; last outcome recorded in
GITHUB_SYNC_LAST_STATUS.
Drive ingester
- Query limited to
'me' in owners, and only files under a canonical domain folder are ingested (it previously scanned every Doc visible to the account, including docs shared by others). - Tags go in the Drive description only; Google Doc bodies are no longer modified with a YAML block.
- Retired
gemini-2.0-flash/gemini-1.5-flashendpoints replaced bygemini-2.5-flash-lite; marker idempotency; sharding.
Tooling
- Jest ignores
.claude/worktrees (every suite ran once per worktree).
Testing
- New
tests/phase0-hardening.test.js(bothcode.gsandsrc/index.js) plus updated drive-ingester tests. npx jest --runInBand --coverage: 25 suites, 1220 tests passing; coverage 97.8 / 87.09 / 98.98 / 99.05 (thresholds met).npm run check(prettier, eslint, PII) passes;code.gs↔src/index.jsparity verified.
Summary by CodeRabbit
- New Features
- Drive and email activity can be organized into monthly or quarterly notes.
- Email filing better recognizes payment receipts, signature documents, career-related messages, and financial security alerts.
- Email note creation supports excluding selected sub-labels while retaining Gmail filing.
- Drive summaries include a source reference, and Drive tagging appears in file descriptions.
- Bug Fixes
- Drive ingestion is limited to owned, non-trashed documents in recognized folder trees.
- Duplicate email and Drive entries are less likely to be added again; existing GitHub notes are protected when their content cannot be read safely.
- Email recovery searches skip Promotions, Social, and Forums; classification matching is more precise.