fix(chat): assemble stable prompts and preserve branch context

Merged
WE
11h ago
+2357-570

Description

Build chat prompts from a stable preamble, chronological history and current-step reminders. Keep agent instructions, project/persona text and images ahead of conversation history. Keep citation guidance for saved sources without replaying unrelated old reminders. Deep Research retains persona images and places its changing cycle count after history.

Preserve the selected conversation branch in temporary, usage-only sessions. Each answer owns its compaction checkpoint, including side-by-side answers. Follow-ups reuse stored attachment text. Missing temporary answers do not block new turns or substitute another model's answer. Evict complete turns: attachments, questions, tool exchanges and answers leave together. An oversized current turn fails atomically.

This is the third layer of the stack. The lower layers own tool-choice behavior and retaining tool results. The follow-up layer protects tool configuration across runtime steps.

Temporary records from before this change lack response ownership and expire on upgrade. New turns continue without that old context; old questions cannot be regenerated. Teardown fences remain valid across the upgrade.

Cache expiry, changed settings/files/models, image limits, research phase changes and compaction can still invalidate prefixes. This removes avoidable prompt rewrites; it does not guarantee savings for every conversation.

Measured cache and cost profile: cold no-tool caching cost 25% more; keeping final tool schemas saved 43% across four calls; retaining evidence saved 41% across eight steps but cost 4% more in a short trace. The report includes raw provider usage, reproduction scripts, pricing and limitations. These are controlled measurements, not production averages.

OpenAI measurements are also available: raw usage, scripts and limitations. GPT-6 Luna reused 6,327 cached tokens through the Onyx provider path. GPT-4.1 mini retained-history costs were 24% lower in the eight-step trace, with one additional arithmetic error. Cold-cache costs and the separate existing cache-write accounting gap are documented.

How Has This Been Tested?

  • 2,745 unit tests passed with the runtime guardrail layer across chat, agents, Deep Research, context, LLMs, file storage and history replay.
  • 54 real Redis tests passed, including whole-turn eviction, attachment groups, sibling checkpoints and atomic cap rejection. All six new retention regressions fail against the previous implementation.
  • Connected PostgreSQL/Redis and answer tests cover compaction, restoration, attachments and failed response saves.
  • Live Anthropic and OpenAI smoke tests passed. Earlier frontend/API runs verified DOCX and image context through initial answers, follow-ups and regeneration.
  • Live Anthropic history replay recalled an unrevealed tool-result value; the follow-up read 8,059 cached tokens and wrote 61. An omitted-result comparison answered UNKNOWN and read no cached tokens.
  • Commit checks passed, including types, lint, feature-map integrity and docs checks.

Docs

  • Customer-facing change: I updated documentation/ (text and screenshots).
  • No docs change needed (internal, tests, CI, or multi-tenant cloud only).

Updated the existing FAQ, chat/runtime documentation and feature maps. No UI changed; screenshots are not needed.

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.

Summary by CodeRabbit

  • New Features

    • Incognito chats retain attached text files and images for follow-up questions during the session.
    • Deep research can use request-specific context and attached images.

  • Improvements

    • Follow-up prompts better preserve conversation history, attachments, and action results when continuing or regenerating from an earlier point.
    • Project instructions, files, and images are arranged consistently in prompts, while request-specific context stays separate from saved history.
    • Citation guidance appears when earlier responses contain citations.
    • Research prompts clarify that completing every available research cycle is optional.
    • When conversation summaries cannot be created or used, older history is handled more reliably.

  • Documentation

    • Updated guidance on chat history, Incognito attachments, and prompt caching.