#181 feat(skill-memory): per-skill cross-session recall + historian auto-extraction

Summary

Adds skill-memory — per-skill "motor memory" that gives a skill cross-session recall of its own hard-won lessons (gotchas, discoveries, fixes, workflow steps). When a skill's SKILL.md declares skill-memory: { enabled: true }, accumulated notes for that skill surface automatically in a <skill-memory> block appended to the skill tool's result on every load — and, with the historian extension, the historian writes those notes automatically during compaction (no agent action required).

It's fully opt-in per skill and cache-safe by construction: the block rides the tool-result tail (conversation), never the cached system/m[0] prefix, so it can't bust the prompt cache.

How it works

• Transparent recall — three-hook augmentation: tool.definition advertises an intent param; tool.execute.before stashes the intent (bounded TTL); the after-hook parses the skill's Base directory, reads its SKILL.md frontmatter, and formats the recall block. Lands in the tool RESULT (cache-safe).
• Write paths — ctx_skill_note (agent-authored) and the historian (auto-extracted). Both dedup on a normalized hash.
• Intent-scoped ranking — a recall cascade: model-matched embeddings → cosine blend over intent_embedding + delta_embedding (relevance/recency/hit weights tunable per skill via ranking_* frontmatter); FTS5 fallback over a content-linked skill_memory_fts vtable; flat recency×hit fallback.
• Historian auto-extraction — the historian sees TC: skill(<name>) markers in its chunk and emits a <skill_observations> block; both the OpenCode and Pi runners promote those post-commit as global notes (project_identity='*', source_type='historian'), recallable from any project.

Review units (4 commits)

The branch is organized into four coherent, reviewable phases:

1. P1 — transparent per-skill recall: skill_memory table migration, the three-hook augmentation, flat recall + storage, ctx_skill_note/ctx_skill_recall tools, opt-in distill-skill-memory dreamer task, TUI/ctx-status stats, docs.
2. P2 — embeddings + intent-scoped recall: delta_embedding + recall_count columns + skill_memory_fts FTS5 vtable; embed-on-write; the cosine/FTS recall cascade; a programmatic (no-LLM) reembed pre-step.
3. P3a — historian-extraction foundation: the TC: skill(<name>) marker keystone; origin_project + source_type columns; global-tier notes unified under project_identity='*' (collision-merge); partitionKey routing.
4. P3b — historian auto-extraction pipeline: prompt emits <skill_observations>, parser, validated-result threading, both runners promote via the shared promoteSkillObservations helper, plus an initializeDatabase self-heal net (re-creates skill_memory + ensureColumn so an upgraded DB recovers even if a migration row is lost).

Schema / migrations

Three migrations (numbered after the current master ceiling): skill_memory table, then delta_embedding/recall_count/FTS, then origin_project/source_type/* unification. LATEST_SUPPORTED_VERSION bumped in lockstep (the schema-version-fence test enforces it). Migration bodies are ensureColumn/IF NOT EXISTS idempotent.

Testing

Full plugin + Pi suites pass (one unrelated pre-existing full-suite ordering flake in tui-config.test.ts that passes in isolation), tsc clean both packages, lint clean, build produces all bundles. Dedicated coverage for the migrations (coexistence + fence), recall rungs, the tools, FTS triggers/backfill, the '*' collision-merge, and the historian promotion path on both runners. Verified working live: the historian auto-extraction writes genuine source_type='historian' notes, embeddings populate, and the read-side recall_count increments on surfacing.

Notes for reviewers

• Migration version numbers are placeholders relative to this fork's base — happy to renumber to whatever slots are free at merge time.
• The four commits are independently meaningful; if you'd prefer this as stacked PRs (P1 / P2 / historian), I can split it.

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

Summary by cubic

Adds per-skill cross-session recall and historian auto-extraction. Previously skills had no recall; now opted-in skills append a cache-safe <skill-memory> block on each load, and the historian writes global notes so skills remember gotchas, discoveries, fixes, and workflows without busting the prompt cache.

• Transparent recall: the skill tool exposes an optional intent; hooks stash it and append a rendered recall block. Ranking cascades from cosine over on-the-fly intent + stored delta_embedding, to FTS5, then recency×hit. Reads union the skill’s project partition with the global '*' tier; reads increment recall_count. Token budgeting counts per-note framing; pinned budget clamps to min(max_pinned_tokens, max_tokens).
• Write/recall tools: ctx_skill_note dedups by hash and cosine, embeds on write, and rejects writes when a skill hasn’t enabled memory. ctx_skill_recall renders notes on demand. Both are OpenCode-only; Pi does not register them.
• Historian auto-extraction: the historian emits <skill_observations>; both OpenCode and Pi promote them as global '*' notes (source_type='historian') via a shared helper, gated to avoid double-emits.
• Provenance and safety: accepts plain paths and file://; Windows path normalization; name-based fallback when the trailing provenance line is truncated. Intent stash is keyed by sessionId:callId with TTL and per-session pruning. The skill tool’s model schema advertises the intent param via jsonSchema. ctx_skill_note now fails early when a skill hasn’t opted in.
• UX and surface: async executeStatus and the TUI show per-project skill-memory totals. The system prompt adds skill-memory guidance. The A1 golden and prompt-surface budget fixture were re-measured to include ctx_skill_note and ctx_skill_recall under the existing policy.
• Dreamer: adds opt-in distill-skill-memory; a pre-step re-embeds NULL/stale vectors, then emits a read-only corpus health report.

Rollout

• Fork-lane migrations v10000–v10002 add skill_memory, embeddings + FTS, and historian provenance/global '*' unification. initializeDatabase self-heals (creates table/columns, rebuilds FTS) and runForkMigrations runs on DB open.
• Required action: enable per skill by adding skill-memory: { enabled: true } to SKILL.md. Optional: schedule the distill-skill-memory dreamer task.

Confidence Score: 5/5

• This PR is safe to merge. The core recall, storage, and historian promotion paths are well-implemented and previously flagged bugs have all been addressed.
• The three bugs identified in the previous review round (session-bleed on intent stash, FTS rebuild gap in self-heal, orphaned historian notes for project-local skills) were all fixed. The remaining findings are a misleading parameter name on stashIntent and a redundant double-call inside searchSkillMemoryFts — neither affects correctness at runtime.
• No files require special attention beyond the stashIntent parameter naming in hook-handlers.ts, which is a latent readability concern rather than an active defect.

sequenceDiagram
    participant Agent
    participant SkillTool as skill tool
    participant BeforeHook as tool.execute.before
    participant AfterHook as tool.execute.after
    participant Registry as SkillLoadRegistry
    participant DB as skill_memory DB
    participant Recall as recallSkillMemoryBlock

    Agent->>SkillTool: skill(name, intent)
    SkillTool->>BeforeHook: fires pre-validation
    BeforeHook->>BeforeHook: stashIntent(map, sessionId:callId, intent)

    SkillTool-->>AfterHook: output with "Base directory for this skill:"
    AfterHook->>AfterHook: parseSkillProvenance(output)
    AfterHook->>AfterHook: parseFrontmatterConfig(SKILL.md)
    AfterHook->>Registry: set(sessionId:skillId, provenance + frontmatterConfig)
    AfterHook->>AfterHook: getAndDeleteIntent(map, sessionId:callId)
    AfterHook->>Recall: "recallSkillMemoryBlock(db, {skill, intent, scope, projectIdentity})"

    alt has intent + embeddings
        Recall->>DB: getRankingCandidates (200)
        Recall->>Recall: rankRung1(cosine over intent+delta vectors)
        Recall->>DB: getPinnedNotes
        Recall->>Recall: unionPinnedFirst + budgetFill
    else FTS fallback
        Recall->>DB: searchSkillMemoryFts(sanitized intent)
    else flat fallback
        Recall->>DB: getSkillMemoryNotes(recency×hit)
    end

    Recall->>DB: bumpRecallCountByIds(surfaced note ids)
    Recall-->>AfterHook: "<skill-memory> XML block"
    AfterHook->>AfterHook: "output.output += block"
    AfterHook-->>Agent: augmented skill tool result

    Note over Agent,DB: Agent may call ctx_skill_note to write a note
    Agent->>DB: ctx_skill_note(skill, intent, kind, delta)
    DB-->>Agent: "Skill note saved (id=…)"

    Note over DB: Historian compaction (async)
    AfterHook->>DB: "promoteSkillObservations(global, '*', observations)"

_{Reviews (38): Last reviewed commit: "chore(skill-memory): rebase onto upstrea..." | Re-trigger Greptile}