feat(LUKE-48): build Luke's attention intelligence

Merged
DA
2mo ago
+1904-0

Summary

  • Add a portable attention layer in packages/sidecar-core/ that derives a bounded, redacted update only when a session reports something new, validates untrusted model output against a small decision contract, deduplicates repeated speech per session, bounds how many updates one pass evaluates, and keeps one evaluation in flight per session.
  • Add an OpenAI evaluator in the Electron main process that calls the Responses API with the decision contract as a strict structured-output schema, sets store: false, and returns nothing when the API is unavailable or answers outside the contract.
  • Wire the reviewer into session observation outside the refresh guard, so a slow model call never delays the next provider snapshot, and store each decision on the session without touching provider-owned data.
  • Discard a decision the session has already moved past, so answering a waiting session while the model is still thinking never produces a stale interruption, and keep that development derivable for a later pass.
  • Ship redacted, synthetic tuning examples that serve as both the prompt's few-shot guidance and its regression coverage.

Decision contract for consumers

AttentionReview answers two different questions and they must not be conflated:

  • decision — whether the session warrants attention and how urgently. This is what callers store on the session, and what the renderer reads.
  • outcome — whether Luke should voice it now: decided, deduplicated, superseded, or unavailable.

A repeated development carries a speaking decision with a deduplicated outcome: the session still needs attention, but saying the same sentence again would be noise. A consumer that infers speech from decision.disposition alone will repeat itself — read outcome for speech.

Trust boundary

  • The layer is optional. Without OPENAI_API_KEY, Luke observes sessions and stays silent; no product behavior requires credentials.
  • The only session material sent is the bounded update: provider name, session title, trigger, previous status, status, and the provider's redacted summary. No transcript, file path, or command output is ever included, and evidence/capture mode never enables the layer.
  • Every failure path — network error, non-2xx status, non-JSON body, unknown disposition, a speaking decision with no sentence, a session that moved on or vanished mid-evaluation — resolves to silence rather than a guess. Each review reports which of those happened through an outcome value set.
  • Logs carry a status code or an error name only, never the request, the key, or session material.
VariableDefaultPurpose
OPENAI_API_KEYunsetEnables attention review when present
LUKE_ATTENTION_MODELgpt-5.6-lunaModel used for the decision
OPENAI_BASE_URLhttps://api.openai.com/v1Alternate OpenAI-compatible endpoint

Evidence

  • Platform-independent checks: ./scripts/check.sh passed (lint, typecheck, 26 desktop tests, 31 core tests, 5 harness tests, all builds)
  • macOS Electron verification (./scripts/verify.sh): portable checks passed, then packaging stopped with error: this command requires macOS in this Linux workspace; CI's macOS app/evidence job covers it
  • End-to-end run of the real pipeline (InMemorySessionRegistry → SessionAttentionReviewer → OpenAiAttentionEvaluator over real fetch) against a local stub of the Responses API:
first observation:            reviews=1  session-a=silent
unchanged:                    reviews=0                       <- no API call
became waiting:               reviews=1  session-a=speak-during-turn
resumed working:              reviews=1  session-a=silent
waiting again (deduplicated): reviews=1  session-a=silent (suppressed)

requests sent: 4
path: /v1/responses
body keys: input, instructions, max_output_tokens, model, store, text
store: false
input sent to the API:
---
Provider: Claude Code
Session: Claude Code: checkout-service
Trigger: observed
Previous status: none
Status: working
Observed summary: Claude Code working; transcript content is not retained.
---

Automated visual evidence

Download the deterministic macOS evidence · workflow run

  • Commit: 5a2d610324c08fc4d6730971ec6a4a2fce7de57b
  • Scenario: smoke
  • Physical-notch check: not performed by CI

Physical-device evidence

  • Screenshot or screen recording: not attached — no renderer or window change in this PR
  • Physical-notch check: not performed
  • Device/display configuration: not recorded

Notes

  • No live OpenAI credential is available in this workspace, so the API contract is exercised against a local stub and unit-level fakes rather than the hosted API. The hosted API's acceptance of the strict schema is the one surface left untested.
  • The default model is gpt-5.6-luna (04c5d9e). The task is a three-way classification against a fixed prompt run in the background on the user's own key, so the cost-optimized tier fits: it is ~25x cheaper per call than the flagship, and its lower latency means fewer decisions are discarded as superseded. LUKE_ATTENTION_MODEL overrides it.
  • The renderer is unchanged. It already treats a non-silent disposition as a session needing attention, so decisions surface through the existing indicator with no UI diff.
  • Merged origin/main (8ccaa5c) after #9/#10/#11 landed. GitHub reported no conflict, but #11's new fixtureMode created a semantic one: the attention evaluator still keyed off captureMode, so a deterministic --fixture run would have read OPENAI_API_KEY. Fixed in 5a2d610, and ./scripts/check.sh was re-run green against the merged tree.
  • A development is consumed only once a decision is actually reached about it. A superseded decision stays derivable, and an unavailable evaluator is retried twice per session before the update is dropped — enough for a passing network blip, bounded so a standing misconfiguration cannot re-evaluate on every poll.
  • Cursor Bugbot found three real defects across review rounds, all in the same area — a decision outliving the state it was made about. Each fix has a regression test that fails against the preceding commit:
    • 24b57fa — a speak decision could land after a provider refresh moved the session on. Re-check the session against the evaluated state before admitting a speaking decision.
    • 77af446 — that check ran per-evaluation, but waiting on a slower sibling in the same pass reopened the window. Settle all speaking decisions in one uninterrupted step after the last evaluation lands.
    • ca60e6f — a superseded settle undid the ledger write but not the baseline advance, so a session that returned to the evaluated state was never reviewed again. Restore the pre-pass baseline when a decision is dropped as superseded.
    • 5955b0e — deduplication downgraded a repeat to silence, and the renderer reads the stored disposition, so a second turn finishing minutes after the first was hidden entirely. Keep the disposition and report the repeat through outcome.
    • 3bd09fb — reasoning tokens share the max_output_tokens budget, so a 1024 cap could return incomplete with no output; consuming the baseline on failure then made that permanent. Raise the cap and retry an unavailable evaluation twice.
    • aba024f — the retry budget did not reset on a superseded answer, so sparse blips could accumulate into a dropped development. Clear the streak on any outcome except unavailable.
    • 951684b — restoring the pre-pass baseline only re-derives an update while the session sits elsewhere, so a complete → working → complete round trip vanished. Forget the observation instead.
  • Follow-up: speaking the decision aloud belongs to the realtime voice work, not this PR.

Open workspace in Conductor