perf(chat): load a chat a page at a time

Merged
RA
13h ago
+439-7

Description

Opening a chat fetched every message on every branch, with all of their replay packets. Render windowing (#17021, the base of this PR) bounds what the browser mounts, but not what it downloads and parses. A seeded 500-turn chat was a 22 MB response, and a 1,000-turn chat 95 MB, before the first message could show.

Backend

  • GET /api/chat/get-chat-session/{id} takes two optional query parameters, limit (1–200) and before_message_id. With limit, it returns:
    • the last limit messages of the current branch (the chain of latest_child_message_id from the root), or of the part before before_message_id;
    • every child of those messages, so the branch switcher and multi-model replies have their siblings, including the siblings of the first message of the next, newer page;
    • has_older_messages, set when the branch has messages before the page.
  • get_chat_message_page_ids (onyx/db/chat.py) finds the page from a light query of the session's ids and links. get_chat_messages_by_session takes a message_ids filter, so the replay builds packets for the page only.
  • Without limit, the endpoint returns the whole session as before. The export, the shared view and the readiness check still use it that way.

Frontend

  • The chat loads its last 20 messages (lib/chat/pagedSession.ts).
  • OlderMessagesLoader: a message skeleton above the loaded messages. When it nears the viewport, it loads the next older page and merges it into the session's tree.
  • A page added above keeps the view still. ChatUI scrolls by the distance the former first row moved. Native scroll anchoring is not enough: Safari has none, and Chrome skips it at the very top, where a page most often arrives.
  • Switching to a branch whose messages are not loaded reloads the last page once the server has the new branch.

Docs

  • Customer-facing change: I updated documentation/ (text and screenshots).
  • No docs change needed (internal, tests, CI, or multi-tenant cloud only).

Additional Options

  • [Optional] Please cherry-pick this PR to the latest release version.

Summary by cubic

Opening a chat now loads only its last 20 messages instead of the entire session, so long chats show content without downloading every branch's messages and replay packets (a 500-turn chat used to arrive as a 22 MB response, a 1,000-turn chat as 95 MB).

Backend

  • get-chat-session accepts optional limit (1–200) and before_message_id, returning the last limit messages of the current branch (or those before a message), every child of those messages so sibling branches and multi-model replies stay available, and has_older_messages. A before_message_id no longer on the current branch returns an empty page, so an earlier request is not repeated in a loop.
  • Paged loads reuse get_chat_messages_by_session with a message_ids filter so replay packets are built for the page only.
  • Without limit, the endpoint returns the whole session as before; the export, shared view, and readiness check are unchanged.

Frontend

  • OlderMessagesLoader, a message skeleton above the loaded messages, loads the next older page as it nears the viewport and merges it into the session's tree; a page that arrives after the tree was replaced is dropped, and a failed load stops loading older pages until the chat is reopened.
  • An empty page means the oldest loaded message is no longer on the server's branch (for example, switched in another tab); the chat then loads that branch's last page.
  • The UI keeps the view still when a page lands above by scrolling by the distance the former first row moved; native scroll anchoring is unreliable (Safari has none, Chrome skips it at the very top).
  • Switching to a branch whose messages aren't loaded reloads the last page once the server has the new branch, and skips the reload if the chat changed while it loaded (for example, a message was sent).

Summary by CodeRabbit

  • New Features
    • Chat history now loads in pages, fetching older messages as you scroll upward.
    • The message list preserves its scroll position as earlier messages load.
    • Chat sessions loaded without pagination continue to show all branches.
    • A loading indicator appears while earlier messages are being fetched, with accessible labels in supported languages.
    • After saving a message, the latest chat page refreshes when needed to show the updated conversation.