feat(chat): draw Mermaid diagrams, preview HTML and copy tables in replies

Merged
AV
11d ago
+2830-144

Links

Summary

  • A closed ```mermaid fence in an assistant reply draws as a diagram on the reply itself, with show code, expand and copy above it, a lightbox with zoom and pan, and the code (plus the parser's whole message) when it cannot be drawn.
  • A closed ```html fence gets a Preview/Code toggle; the preview runs in an <iframe sandbox="allow-scripts"> served from a superset-preview: scheme with its own Content Security Policy, sized from the page, with a zoom menu and full screen.
  • Every table carries Copy as Markdown and Copy as CSV.

Why / Context

Agents answer with diagrams, small HTML mock-ups and tables, and the chat showed all three as text. Diagrams had to be pasted elsewhere to read, HTML had to be saved to a file, tables had to be retyped.

How It Works

  • packages/chat-ui holds the desktop-agnostic pieces: MermaidDiagram (hook with a per-source cache and a renderer seam, drawMermaid on pele, sanitizeSvg on DOMPurify, token mapping CSS, DiagramLightbox), HtmlPreview (frame, previewDocument, zoom, dialog), ChatTable with its copy actions built on Streamdown's table helpers, and ViewToggle.
  • packages/shared/src/html-preview.ts defines the scheme, the policy and the URL codec. apps/desktop/src/main/lib/html-preview-protocol.ts serves the document with the policy as a response header (nosniff, no-referrer, no-store, 413 above 1 MB); main/index.ts registers it and the renderer's frame-src gains exactly that scheme.
  • The desktop chat composes them in ChatCodeBlock: MermaidBlock, HtmlPreviewBlock, PlainCodeBlock. An open fence shows code (Streamdown's useIsCodeFenceIncomplete; ChatMarkdown passes isAnimating for an open fence), a closed one draws or previews.
  • Renderer: pele 1.10.0 (MIT, no dependencies), one chunk per diagram type, colours through --pele-* variables mapped to the app tokens so a theme switch recolours in place. 1.12.0 is inside the repo's minimum release age, so 1.10.0 is pinned.

Manual QA Checklist

Done live in the worktree's dev app over the DevTools protocol (shots on the evidence page):

  • Eight diagram types draw (flowchart LR/TD, sequence, state, class, pie, ER); a broken fence falls back to the code with the parser message
  • Code shows while a fence streams; the diagram appears once it closes
  • Theme switch recolours a drawn diagram in place; light theme and a 256px pane checked
  • Lightbox: fit, wheel zoom around the pointer, drag, buttons, Escape returns focus
  • HTML preview: Preview/Code toggle, height from the page, zoom Fit + 4 steps, full screen, Escape
  • Sandbox, evaluated inside the frame: parent.document, top.location, window.open, fetch, remote img, cookie/localStorage all blocked; window.App/ipcRenderer/electronTRPC/webUtils undefined; the page's own script runs
  • Copy as Markdown / Copy as CSV: clipboard read back after each click
  • Reviewer: open a chat, ask for a mermaid diagram, an html page and a table, and try the three blocks

Testing

  • bun test in packages/chat-ui (52 pass), packages/shared html-preview (3 pass), desktop ChatCodeBlock + main/lib/html-preview-protocol (36 pass)
  • bun run typecheck in packages/chat-ui, packages/shared, apps/desktop: exit 0 each, on the tree after merging main
  • bun run check:i18n: exit 0; every new string has a hand-written translation in the 16 locales
  • bunx biome check on the changed paths: clean
  • Not run locally: the full bun run lint, lint:check-node-imports, desktop build. CI covers them.

Design Decisions

  • A scheme of its own instead of srcdoc or data:: in the same renderer both inherit the app's policy, which has no 'unsafe-inline' for scripts, so a previewed page could not run. A document with its own response headers is the only way to give the frame its own policy.
  • pele instead of mermaid.js: the first pass used mermaid.js and the result read poorly on dark backgrounds (fills within a few percent of the background, pie slices in greys, 16px labels wrapping at 200px). pele colours through CSS variables and lays out from the page font.
  • No card and no height cap around a drawn diagram: a diagram is already a set of boxes; the card added a second border, and a 480px cap left tall flowcharts unreadable until the lightbox opened. The card remains for the code view, open fences and errors.
  • Streamdown helpers used, its controls not: the table and Mermaid controls hang off wrappers that the chat's table and code overrides replace, and its inline Mermaid block captures the wheel for zoom, which would stop the transcript scrolling.

Known Limitations

  • No will-frame-navigate guard yet: the sandbox refuses top navigation and popups, but a previewed page can still navigate its own frame to a remote URL.
  • The comment pane and the Markdown file viewer still draw with mermaid.js through packages/ui.

Follow-ups

  • Copy or save a diagram as an image (SUPER-2131); Retry when the renderer chunk fails to load; a cell-wrap toggle for wide tables.

Co-Authored-By: Claude noreply@anthropic.com

Summary by cubic

Agent chat replies now draw Mermaid diagrams and preview HTML mock-ups in place, and every table can be copied, instead of showing all three as text.

Features

  • A closed ```mermaid fence draws as a diagram on the reply with a Code/Diagram toggle; clicking it opens a lightbox with zoom and pan. An unparseable diagram falls back to its code with the parser's message.
  • A closed ```html fence gets a Preview/Code toggle. The preview runs in a sandboxed iframe served from a new superset-preview: scheme with its own CSP, so the page's scripts run while network, navigation and app APIs stay blocked.
  • Every table carries Copy as Markdown and Copy as CSV. A multiline cell renders as <br> in the copied Markdown; a refused clipboard write says "Copy failed".

Notes

  • Diagrams render with pele, which takes colours from CSS variables, so a theme switch recolours in place; pele 1.10.0 and dompurify are new dependencies.
  • The main process serves the scheme; the renderer CSP gains it in frame-src only, and malformed preview payloads are rejected without throwing.
  • The lightbox and the full-screen preview share a useModalOverlay hook: the rest of the body is inert, focus moves in, Escape closes the view (forwarded from inside the preview page through a source-checked message), and focus returns on close.
  • A previewed page cannot navigate its own frame off the preview scheme; will-frame-navigate refuses those navigations.

Summary by CodeRabbit

  • New Features
    • Chat code blocks now render Mermaid diagrams and complete HTML snippets as interactive previews, with controls to switch views, adjust zoom, and expand diagrams.
    • Chat tables offer actions to copy their contents as Markdown or CSV.
    • Code blocks display language or file labels and provide copy and line-wrapping controls.
  • Bug Fixes
    • Oversized HTML previews now show a message instead of attempting to load.