feat(desktop): three-arm new-workspace prompt form-factor experiment
Reworks the new-workspace sample prompts so the prompt-cards experiment measures form factor and nothing else, and adds a third arm to test whether four cards cost more than they earn.
Why this is safe to change
The experiment (PostHog 419092) never launched — still draft, flag active: false. All 639 $feature_flag_called evaluations since desktop-v1.22.0 returned null, so nobody has ever been bucketed into an arm. Zero experiment data exists, and #6537 / #6540 are in no shipped desktop tag.
What had drifted
Pre-registration said control = today's three inline rows, unchanged; test = two cards. On main:
| Pre-registered | On main before this PR | |
|---|---|---|
| control | 3 fixed rows | 4 shuffled from a pool of 6 |
| test | 2 bordered cards | 4 borderless cards, shuffled |
| selection | identical across arms | setup-first in cards only |
That last row is the blocker: the cards arm got a curated setup-first list and rows got a random slice, so the arms differed in content inside a layout test.
Premises, validated against production
- Setup is 69.5% of clicks, not 75% — but it always renders first, so share and position are perfectly confounded.
- Setup clickers create ~3x more workspaces (medians), and only 12.7% of those predate the click. But the cold-start cell is n=17, and
fix-small-bug— the least-clicked prompt — shows the highest count, which is a selection signature. Suggestive, not established. - Eligible population is ~840 new accounts/week at a stable 13.5–14.1% click rate. Activation can't resolve a realistic form-factor effect in under ~10 weeks; click rate resolves a large relative effect in ~2–3 weeks across three arms.
So click rate becomes the primary metric and activation is demoted to directional. Powering this on activation is what made a 3-arm test look unaffordable.
Changes
- Fixed five-prompt pool, curated order. Arms slice nested prefixes (
cards2⊂control⊂cards4), so only form factor varies.clean-up-todoscut — with fixed order and nesting, only five prompts are ever reachable. - Shared
useSamplePromptSelection— setup-first rule and its pending gate now apply identically to every arm. Previously cards held for the setup query while rows rendered instantly, a timing difference the experiment would have read as a layout effect. - Three arms:
control(3 rows) /cards2/cards4.layouton the click event distinguishes them. - Card border restored — prominence is the manipulation under test; #6540 had slid it to borderless across three commits.
- Shuffle dropped — randomizes content against a tight power budget and breaks the nesting.
- Dismiss gated behind the user's first non-
mainworkspace. It permanently removes the measured surface, and asymmetric dismissal would bias the primary metric toward null. (mainis auto-created for every account, hence the!= maincondition.)
Kept from #6537/#6540: the expanded pool (a 4-card arm can't be filled from three prompts), per-card icons, AgentLogoCluster, and the labeled model/effort pills.
Before this can launch
- Rename the flag's variants
control/cards2/cards4at 33/33/33 and re-register the experiment with click rate primary. Not done in this PR — no unapproved tracker writes. - Bump the flag's
created_atcutoff to the carrying build's publish time (currently2026-08-17T07:00Z, already in the past).
@Kitenite — this reworks parts of #6537 and #6540, so please take a look. The reasoning is measurement-driven rather than taste-driven; your cards and icons are kept.
bun run typecheck, ./scripts/lint.sh, and 2583 desktop tests pass.
Summary by cubic
Aligns the new-workspace prompt experiment to measure form factor only, adds a third arm, and introduces tiered prompts keyed on whether the user has shipped a workspace. Old behavior randomized content and timing with one cold-start pool; new behavior fixes order, shares selection and timing across arms, restores card borders, distinguishes cards2 vs cards4, and switches between first-run and returning tiers to avoid repeating cold-start prompts.
- Arms: control shows 3 rows;
cards2shows 2 cards;cards4shows 4 cards. All arms slice nested prefixes of one ordered tier list; the setup prompt drops if the project is configured. - Shared
useSamplePromptSelectionapplies the same setup-first rule and pending gate to every arm, so rows no longer appear earlier than cards. - Tiering:
first-runkeeps orientation;returningaddsexplain-supersetand restoresclean-up-todos. Click events carrylayout(rows/cards2/cards4) andtier. - Shuffle is dropped; cards regain borders; dismiss events use the same
layout. The dismiss button appears only after the user has a non-mainworkspace. - Flag reading accepts
cards2/cards4variants; the manual override maps tocards2.
Rollout
- Rename the flag’s variants to
control,cards2,cards4at 33/33/33 and re-register the experiment with click rate as the primary metric. - Bump the flag’s
created_atcutoff to the carrying build’s publish time.
Summary by CodeRabbit
-
New Features
- Added updated workspace suggestion layouts, including two- and four-card formats.
- Suggestions now adapt to project setup and whether you’re creating a workspace for the first time or returning.
- Added an accessible dismiss button for hiding suggestions.
-
Bug Fixes
- Improved prompt loading behavior to prevent incomplete suggestions from appearing.
- Dismissal is available only when appropriate and accurately tracks the displayed layout.
- Updated prompt cards with refreshed styling and clearer actions.