PR #7726 - superset-sh/superset

What

page.list selected every page in the organization with no LIMIT, then built a presigned URL per row in a Promise.all. Query and response both grew linearly with org size.

It now accepts search, limit and cursor, and returns { items, nextCursor }.

Why a keyset cursor and not an offset

The list is ordered by updatedAt, so a republish during a sweep reshuffles it and a row offset skips rows. The keyset is (updatedAt, id), matching ORDER BY updated_at DESC, id DESC.

Ordering is unchanged, so the existing pages_organization_id_updated_at_idx on (organization_id, updated_at DESC) still serves it. No migration.

Why the cursor carries text, not a Date

pages.updated_at is timestamptz, written by defaultNow() (Postgres now(), microseconds) on insert and $onUpdate(() => new Date()) (JS, milliseconds) on update. A JS Date cursor truncates to milliseconds, so every row whose real timestamp fell inside the truncated microsecond satisfied neither < nor = and vanished from every batch.

The cursor now carries updated_at::text and compares against ::timestamptz, which is exact and still index-friendly — unlike date_trunc in ORDER BY, which would have cost a seq scan.

Search

ILIKE over title and slug. LIKE metacharacters are escaped, so --search "100%" doesn't match everything.

Callers

All five migrated to the envelope:

Caller	Change
`cli pages list`	`--search`/`-q`, `--limit`, `--cursor`
`cli pages comments list`	Drains the cursor — it sweeps every page, so a cap would silently hide comments
`mcp pages_list`	Forwards search/limit/cursor, documents `nextCursor` to the model
desktop `PagesView`	`useInfiniteQuery` + background drain
mobile `usePagesQuery`	Drains in `queryFn`

Desktop and mobile compute tab counts and scope filters over the whole set, so a cap would have made those counts wrong — they drain rather than truncate.

CLI back-compat

superset pages list with neither --limit nor --cursor drains every page and still emits a bare array, so existing --json | jq '.[].id' and --quiet scripts are unaffected. Passing either flag opts into a single batch and the { items, nextCursor } envelope.

cli-framework's extractIds learned to unwrap a paginated envelope so --quiet keeps printing ids rather than raw JSON. That only affects results which today print useless JSON.

Review fixes

A review pass caught these, all fixed here:

Desktop infinite refetch loop. React Query derives hasNextPage from the last successful fetch, so a failed batch left it true while isFetchingNextPage flipped false — re-arming the effect forever. Now gated on isFetchNextPageError.
The microsecond truncation above.
Silent CLI truncation. Resolved by draining unless a pagination flag is passed.
--limit 0 / --limit 500 / --limit 2.5 now raise CLIError client-side instead of being dropped or surfacing a raw zod BAD_REQUEST.
Pinned tab no longer disappears from under a user sitting on it while later batches load.

Testing

typecheck — 41/41 packages clean.
31 new/updated tests pass: schema defaults and bounds, cursor precision retention, cursor round-trip and rejection, --quiet/--json envelope handling.
lint:fix, check:i18n, check:plugins, sherif — clean.
CLI binary builds; all four --limit/--cursor guards verified to fire before any network call.
Full bun test: the only failures are two pre-existing terminal-ws-transport cases (jest.advanceTimersByTime missing in this Bun version), confirmed identical on a clean tree.

Not covered

The keyset SQL has never run against Postgres. *.integration.ts files aren't wired into CI (only test:integration:terminal is) and no DATABASE_URL was available, so the query plan and the ::text/::timestamptz round-trip are reasoned-about, not executed. Worth a run against a Neon branch before merge.

Two deliberate omissions:

plugins/superset/skills/page/SKILL.md documents superset pages list and is still correct given drain-by-default, but doesn't mention --search/--limit/--cursor. Updating it requires a plugin publish + tag, so I left it as a follow-up rather than bump the marketplace unprompted.
Mobile drains serially behind its 30s staleTime, which is slower than the old single request for orgs past 200 pages. Correct but unoptimized; an infinite query there would keep the win.

bun.lock gains one line: cloud-client/package.json already declared @trpc/client and the lockfile hadn't recorded it. Pre-existing drift bun install corrected.

Since the original PR

Reopened from superset-sh/superset — #7529 ran from a fork, so pull_request runs got no secrets and Deploy Database (Neon) failed with Input required and not supplied: api_key before touching a database. Same commits, same branch name, now on origin so preview deploys can authenticate.

Three commits on top of the pagination work, closing SUPER-2357:

52b28df17 — page.list returns workspaceLinks per page, read after the keyset via a batched IN (…) rather than joined into it: workspace_pages is many-to-many, so a join would spend limit + 1 on rows instead of pages and repeat an id across the cursor boundary. Adds the CLI WORKSPACE column and a desktop workspace filter. pages list now also announces the workspace filter it inherits from $SUPERSET_WORKSPACE_ID, which previously made an empty workspace indistinguishable from an empty org.
bd7a80b44 — names cloud workspaces too. listWorkspacesOnHost is single-host by design, so a page published from a sandbox rendered as a bare id.
665399887 — review fixes: prefer a named workspace link over array position, order the links query so workspaceLinks is deterministic, and show the desktop filter at one option rather than two.

Workspace names are resolved client-side throughout. They live in the host's SQLite and in cloud_workspaces, never in a form page.list can join — the server returns ids and each client names what it can.

Note for a reviewer

The cloud_workspaces.name schema comment (schema.ts:749-753) says that column is a provisioning input and "a rename lands on the sandbox and leaves these behind". That is no longer true: cloudWorkspace.rename writes that row, and the desktop deliberately routes a sandbox rename there because "renaming through the host would write to a name nothing reads". The comment predates the change and misleads; left alone here as it is outside this PR.

Still not covered

Unchanged from the original: the keyset SQL has never run against Postgres, and neither has the new IN (…) read beside it. *.integration.ts is not wired into CI. The Neon job above is a deploy step, not a test run, so a green check does not close this.

Summary by cubic

page.list used to return the entire row in an unlimited query per organization, with a single row with fewer posts, etc.

Now, expect a filtering by the actual width.

The system that follows from the introduction. The input copies the older content.

Wait, the one-liner:

I need to read the reader: with `support !

Let me say:

Since the final summary, now write a single sentence.

What is a closed output in the final answer:

The final layering page here is mostly give you this:

The answer is set up to be consistent with the left resources.

Summary by CodeRabbit

New Features
- Desktop page lists support workspace filtering, debounced search, and automatic loading of more results, with server-provided counts and workspace options.
- Mobile page lists load more results as you scroll.
- The CLI and page-list tool support search and cursor-based pagination; CLI results include workspace information. Comment listings can retrieve threads across an organization, with optional workspace and status filters.
Bug Fixes
- Later-batch loading errors preserve pages already shown and provide a retry option.
- CLI quiet output correctly extracts IDs from paginated page-list results.