Groq menu highlights API

NO
6d ago
+11506-18

Adds an optional highlights object to today's hall menus. It holds up to five offerings per meal, picked in the background by openai/gpt-oss-120b on Groq (low reasoning, strict JSON). The upcoming cross-hall Overview tab in 5CDining is meant to be built on it. Not deployed.

The first commit is the Sept 29 – Oct 4 WIP snapshot, kept as history. The three commits after it converge, harden and document it.

Contract (full spec in docs/menu-highlights.md)

  • Highlights are on by default on /v1/menus and /v1/menus/:hall.
  • ?highlights=false returns the original response shape and skips the extra KV read. Any value other than true/false, or a repeated parameter, returns 400 invalid_highlights.
  • highlights.status is one of ready, pending, unavailable (with reason: attempts_exhausted, daily_budget_exhausted, input_too_large or generation_failed), disabled or not_applicable. Every status except ready has meals: [].
  • On ready, each meal has mealIndex, name, items and groups.
    • A group is { title, kind: "single" | "variants", sourceIds, sourceItems }.
    • sourceItems are the original school rows, unchanged, plus id and station.
    • Groups have no aggregate dietary, allergen or nutrition fields.
    • items is a flat convenience list: the first source of each group.
  • IDs such as m1:s0:i2 are positions within one menu version, not dish IDs. A menu edit invalidates the old picks (status returns to pending).

What the app should render

  • Show groups[].title in the order given.
  • For variants, list the choices from sourceItems, and take dietary badges from each source item, never from the group.
  • Link back to the full menu with mealIndex and the source id.
  • For any status other than ready, just show the full menu. Hide highlights UI when the status is disabled.

Configuring Groq

npx wrangler secret put GROQ_API_KEY   # enables highlights
# optional kill switch: set the variable HIGHLIGHTS_ENABLED="false"

Locally, put GROQ_API_KEY=... in the gitignored .dev.vars. Without a key, every hall reports disabled. No public request can trigger generation.

How it works

  • After the hourly snapshot is saved, the collector's Durable Object alarm sends one Groq request per hall-day (all meals at once), at most one request per alarm, spaced at least 65 s apart.
  • Attempts are reserved in DO storage before network I/O.
  • Limits are 3 attempts per menu version and 21 per California day. Retry-After pauses all halls.
  • Results are reused until the menu content hash changes. The hash covers food, model, prompt version and prompt text.
  • Public requests never call Groq. The projection cost was measured at 0.35 ms for all seven halls.

Changes in this PR, after the snapshot

Converged on one pipeline.

  • Kept the v8 "minimal" contract and removed the v5 flat policy, the full grouped/program-component contract, both grouped prompt strategies, the inventory pilot, and their eval scripts and tests.
  • The 9 highlight modules became 3: highlight-selection.ts, highlights.ts and highlight-jobs.ts.
  • One validator now checks both model output and saved records on read.

Validation.

  • Every meal key appears exactly once.
  • Each source belongs to its meal, with no shared ownership. Limits are ≤5 groups per meal and 1–4 sources per group.
  • Single titles always come from the menu.
  • Combined titles:
    • must be plain text;
    • may use only words found in every source name, apart from connectors and the words that distinguish variants (protein, cooking style);
    • must share a dish word.
  • So dietary or safety wording such as "peanut-free", "keto", "vegan" or "GF" must be in every source.
  • A bad combined title splits into source-named singles instead of discarding the hall-day.

Robustness.

  • Retryable failures (429, 5xx, timeout, malformed JSON, failed validation, json_validate_failed) stay pending until attempts run out.
  • 413 maps to input_too_large.
  • Logs carry a short reason, never provider bodies.
  • Stale menus that still have food remain eligible. Menus with no items are not_applicable.
  • Saved results that stop validating are requeued within budget.

Docs. The contract doc is rewritten. Nine overlapping experiment docs are collapsed into docs/highlights-history.md; the originals are archived in docs/reports/.

An independent review (Opus subagent) found five issues, all fixed with regression tests: dietary bypasses in combined titles, halls stuck pending while their menu was stale, dedupe on truncated titles, a doc mismatch for daily_budget_exhausted, and validator changes leaving halls unavailable.

Verification

  • npm run check passes (typecheck plus 136 tests), as do npm run test:runtime (real workerd, SQLite DO, actual alarm delivery) and npm run build.
  • Live wrangler dev with the real Groq key, Oct 23: checked the full contract, opt-out parity, future-date and invalid_highlights behavior. All 68 served sources matched their original menu rows field for field, and a repeated scheduled run made zero extra calls.
  • First live run exposed two real problems:
    • Hoch hit json_validate_failed twice before succeeding. That error was previously terminal.
    • Frary lost the whole hall to one rejected combined title. That led to the split fallback.

Known imperfect (please read)

Selection quality is decent but not curated. Audit of the final prompt (docs/reports/highlight-audit-2026-10-23.md): 68 picks, 62 credible, 5 weak, 1 clear side error. The earlier v9 prompt made 4 clear errors (3 pastries and a station label).

  • Ranking among real mains is uneven. Generic grill and pizza items sometimes beat named programs: McConnell missed its ceviche bar and pad thai bowl.
  • Programs and their components can overlap. McConnell's Oasis program was picked alongside its own steak. Nothing in code knows which rows belong to a program.
  • Protein variants are rarely grouped. Both available pairs (Hoch bao, Frary banh mi) were split, with only one side selected. Combined groups were 0 of 68.
  • Weak staples or sides still appear occasionally. Examples: hard-boiled egg, breakfast sausage, plain quinoa. Thin breakfasts can now be under-filled: Collins got one pick.
  • Labels are assistant judgments, not human-approved, and cover a single day.
  • Frank's Pomona source failed to fetch on every date during testing. That's unrelated to highlights and not investigated here.
  • Inference is bounded, not exactly-once. A crash after Groq accepts a request can cause one repeat call.

Untested here: production deployment, multi-day behavior, and real account token limits under menu churn.

🤖 Generated with Claude Code

[!NOTE]
Medium Risk
Adds optional third-party inference, extra KV reads per uncached menu request, and background job state in the collector DO; misconfiguration or validation bugs could affect API shape or stall highlights without blocking base menus when disabled or opted out.

Overview
Ships optional AI menu highlights for today's halls: up to five ranked offerings per meal, generated in the background via Groq (gpt-oss-120b) and served from KV—public routes never call Groq.

API: Menu responses gain a highlights object (ready / pending / unavailable / disabled / not_applicable) with per-meal groups (single or variants) and unchanged sourceItems. ?highlights=false restores the old shape and skips the extra KV read; invalid values return invalid_highlights.

Runtime: After hourly collection, MenuCollector schedules DO alarms that run highlight-jobs (paced calls, 21/day and 3/version caps, version hashing, retries). Selection logic lives in highlight-selection.ts with strict validation, combined-title rules, and split-to-singles fallback. npm run highlights:audit and expanded runtime tests cover the path.

Docs: README and new guides (menu-highlights.md, highlights-history.md) document the contract, ops, and known quality limits; experiment reports are archived under docs/reports/.

Reviewed by Cursor Bugbot for commit 8ce5a54b0aefa690c99d7dc8e134dfa3bbae0a78. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by cubic

Adds optional highlights to today's hall menus, generated by openai/gpt-oss-120b on Groq in the background and served from KV. Highlights are on by default; ?highlights=false restores the original response shape and skips the extra read. Selection is decent but not curated (62 of 68 live picks credible); ranking among mains, program overlap, variant grouping, and exactly-once inference remain imperfect. Not deployed.

Contract

  • highlights.status is ready, pending, unavailable (with reason), disabled, or not_applicable; only ready carries meals.
  • ready returns up to five groups per meal: title, kind (single or variants), sourceIds, and sourceItems (original rows plus id and station).
  • IDs like m1:s0:i2 are positions within one menu version, not dish IDs; a menu edit returns the status to pending.
  • One validator covers model output and saved records: every meal key exactly once, ≤5 groups per meal, 1–4 sources per group, and plain-text combined titles that may reuse only words present in every source name.
  • Invalid combined titles split into source-named singles instead of failing the whole hall-day.

How it runs

  • After the hourly snapshot, the collector's Durable Object alarm sends one request per hall-day, spaced 65+ s apart, with attempts reserved before any network I/O.
  • Limits are 3 attempts per menu version and 21 per California day; Retry-After pauses all halls.
  • Results are reused until the menu content hash changes; retryable failures (429, 5xx, bad JSON, failed validation) stay pending until attempts run out.
  • Public reads never call Groq (projection cost 0.35 ms for seven halls). Set the GROQ_API_KEY secret to enable; HIGHLIGHTS_ENABLED="false" is the kill switch.
  • Nine experiment modules and docs collapsed into three source files plus docs/menu-highlights.md and docs/highlights-history.md; 136 tests pass under npm run check along with runtime tests.