feat: [AI-8448] count installs from the shell installers, not just npm
Fixes AI-8448.
The dip was measurement, not installs
first_launch is the only install metric. It fires off a marker file rather than any network call from the installer — and that marker was written in exactly one place, packages/opencode/script/postinstall.mjs. Neither install nor install.ps1 wrote it, so once the advertised path moved from npm to altimate.sh/install, those installs stopped being counted. Nothing changed about how many people were installing.
What this does
Both shell installers now write the same marker postinstall.mjs writes, and first_launch carries a new install_method (curl | powershell | npm | unknown) so the recovered volume is separable from npm instead of folded into one number. altimate upgrade on the curl path re-runs install, so curl upgrades become visible too.
Brand-new installs stay is_upgrade: false — that field probes whether ~/.altimate/machine-id existed before this launch:
count(distinct machine_id) where type = "first_launch" and is_upgrade = false
Expect install_method: "unknown" for the first upgrade after this ships — those markers predate the field.
Details that fail silently rather than loudly
- Marker path.
$XDG_DATA_HOME, default~/.local/share/altimate-code, on every platform including Windows.welcome.tsresolves the data dir through Node'sos.homedir()and never consults%LOCALAPPDATA%; a marker written there would be ignored at read time. -Encoding asciiininstall.ps1. The documented entrypoint ispowershell -c "irm ... | iex"— Windows PowerShell 5.1, where-Encoding utf8prepends a BOM..trim()does strip a leading BOM (U+FEFF is JS whitespace), so this was latent rather than broken, butinstall_methodis matched against a fixed allowlist and shouldn't depend on that.unknownversion fallback. An empty marker is deleted unread, so an unresolved version would lose the install outright. That's the statecheck_versionleaves whenever the GitHub API is unreachable.- Write happens after the install dispatch. A version already present (
check_versionexits 0 early) reports no install, and neither does a failed download. - Allowlisted
install_method. A hand-edited or truncated source file readsunknownrather than minting a new dimension. - Source file is consumed on read, including on the empty-marker path, so an orphan can't be attributed to a later install.
- Non-fatal in both installers. A read-only
$HOMEcosts the event, never the install.
Privacy
No new network call and no new identifier. The installers record a version and their own name to a local file; the CLI's existing opt-out gates (ALTIMATE_TELEMETRY_DISABLED, OPENCODE_DISABLE_TELEMETRY, telemetry.disabled) still decide whether anything is transmitted. docs/docs/reference/security-faq.md and docs/docs/reference/telemetry.md are updated to say so.
Tests
Followed the touchpoint set from #1064 (event union → docs → emitter → unit tests → install-script assertions).
test/cli/welcome.test.ts—is_upgradeboth ways,install_methodattribution, allowlist rejection, source-file consumption, empty-marker path.test/install/install-telemetry.test.ts— marker path/fallback/ordering/non-fatality for both installers, no-BOM, plus the ordering invariant below.test/install/postinstall.test.ts— npm writes.install-source.
The load-bearing test is the ordering invariant. is_upgrade is only correct because src/index.ts fires Telemetry.init() unawaited and doInit() yields at await Config.get() before minting the machine-id, so the synchronous banner call on the next line still sees pre-launch state. An await added ahead of that mint would make every install report is_upgrade: true and silently empty the brand-new-install metric without a single existing test failing. The test asserts the machine-id is absent at that instant and present once the promise resolves, so it can't pass vacuously.
598 pass / 0 fail across test/cli/welcome.test.ts test/install/ test/telemetry/telemetry.test.ts test/branding/; typecheck clean.
Verification
install's marker writer was executed directly: XDG override honored, v prefix stripped, unknown fallback, exit 0 on a read-only $HOME. install.ps1 is asserted at source level only — no pwsh on the dev machine, so its runtime behavior rides on CI.
Not in scope
Installs that never launch the CLI remain uncounted, so download→launch conversion is still unmeasurable. That needs a beacon from the install script itself — a new event plus opt-out handling in bash — and is deliberately deferred.
Two pre-existing things noticed but left alone:
welcome.ts:70returns before printing the welcome box whenisUpgradeis false, so the box only ever shows on upgrades. Plausibly intentional (the TUI has its own first-run flow), but it reads backwards for a "welcome" banner.test/altimate/review/telemetry.test.tsredirects$HOMEto keep the suite from minting a machine-id in the developer's real home — but Bun resolvesos.homedir()at startup and ignores laterprocess.env.HOMEmutation, so that protection doesn't currently work. This PR's tests usespyOn(os, "homedir"), the convention already used intest/mcp/discover.test.ts.
🤖 Generated with Claude Code
Summary by cubic
Counts installs from the shell installers and the VS Code extension installer, attributing each install's source in telemetry. Previously only npm postinstall.mjs wrote the install marker, so installs via altimate.sh/install, install.ps1, and the VS Code extension went uncounted; now all installers write the same marker and first_launch carries install_method (curl/powershell/npm/vscode-extension/local, or unknown). Closes AI-8448.
- Both shell installers and
postinstall.mjswrite.install-sourcebefore.installed-versionand publish the version atomically (temp+rename), so a CLI reading mid-write can't misattribute or drop the install;install --binaryreports"local"and both scripts fall back to"unknown"for an unresolvable version. install.ps1now computes its data dir inside the try: under$ErrorActionPreference = "Stop"a nullUSERPROFILEor bad PSDrive previously aborted the installer after the binary was placed but before the PATH write. The marker block became aWrite-InstallMarkerfunction so Pester can AST-extract and execute it, with the suite pinned to Stop semantics so the try/catch tests can't silently pass.welcome.tsreads and consumes.install-source(cleared infinally, and recursive so a directory-shaped file can't pin attribution tounknown), allowlists the five methods, and addsinstall_methodto thefirst_launchevent.src/index.tsruns the welcome banner beforeTelemetry.init()sois_upgradeis structurally correct instead of relying on init's async timing.- Docs note the installers send no telemetry themselves and that the
telemetry.disabledconfig key can be bypassed when telemetry startup runs before config resolves — env vars remain the guaranteed opt-out. - Rollout: first upgrade after this ships may report
install_method: "unknown"for pre-existing markers. Release the CLI before the VS Code extension starts writing markers so the reader exists first.
Summary by CodeRabbit
-
New Features
- First-launch telemetry now records installation methods including curl, PowerShell, npm, VS Code extension, local binary, or unknown.
- Installers record version and source information locally for later telemetry transmission.
- Telemetry respects opt-out settings, and installers do not transmit telemetry directly.
-
Documentation
- Clarified upgrade detection, delivery limits, installer sources, and fallback behavior.
-
Bug Fixes
- Improved handling of missing, invalid, or unwritable installation markers.
- Marker updates are published safely to prevent incomplete telemetry records.
Review fixes (human + 3 bots)
One real bug. install.ps1 computed $dataRoot/$dataDir above the try. With $ErrorActionPreference = "Stop" and Join-Path's provider-qualified path resolution, a null $env:USERPROFILE or an XDG_DATA_HOME naming a bad PSDrive raised a terminating error there — aborting the installer after the binary was placed but before the PATH registry write. Result: installed binary, not on PATH, directly contradicting the block's own "non-fatal" comment. Now everything is inside the try, using [IO.Path]::Combine to keep PSDrive resolution out of it. The old test passed because it only checked that } catch { appeared somewhere; it now pins that the assignments come after try {.
One false claim of mine, corrected. The test header said install.ps1's runtime behaviour "is exercised by the Windows Installer (Pester) CI job". It is not — that job's subprocess tests deliberately stop the installer via -Help/unknown -Version so nothing downloads, and never reach the marker block. It also runs under pwsh, never powershell.exe, so Windows PowerShell 5.1 — the documented entrypoint and the entire reason for -Encoding ascii — is still unexercised anywhere. I wrote that claim from the job name passing, without reading the Pester suite.
To fix the underlying gap, the block is now a Write-InstallMarker function so Pester can AST-extract and execute it the way it already does Test-Checksum. Five new cases run it against a temp profile: byte-exact contents with no BOM, v-strip and unknown fallback, USERPROFILE fallback, no throw on empty USERPROFILE, no throw when the data dir can't be created. That last pair is the coverage that would have caught the bug above.
Other fixes:
- Companion before trigger, all three writers.
.installed-versionis the reader's trigger; writing it first let a CLI starting in between reportunknown, and since writes truncate first, a reader could observe an empty version file and delete it unread — losing the install, not just its attribution. --binaryis attributedlocal, notcurl(that branch setsspecific_version="local", so it misreported both source and version).write_install_markernow takes the method as$1— which also removed an unbound$marker_sourceI'd introduced mid-edit, aset -uabort waiting to happen.- Test isolation: the ordering test now snapshots/clears/restores
OPENCODE_DISABLE_TELEMETRYtoo —doInit()returns before minting if either gate is set. - Docs: the security FAQ no longer asserts unconditionally that the opt-out decides transmission. Env vars are always honoured; the
telemetry.disabledconfig key can be bypassed when telemetry startup runs before config is resolvable, so that caveat is now stated with a pointer to use an env var for a guarantee.
Still not fixed: the config-key opt-out itself
Flagged by CodeRabbit, cubic, and the human reviewer, and it stays open deliberately. The gate is shared by every event emitted from CLI middleware — this PR raises how often it's hit (~30x volume), it doesn't introduce it. Both available in-PR routes are wrong: failing closed emits nothing at all on the middleware path (killing the feature), and reading config here means duplicating the merge + JSONC semantics of config/config.ts. The fix belongs in telemetry init — make Config resolvable there, or adopt a module-wide fail-closed policy. Tracked in the expanded FIXME; needs its own ticket.
Release order
vscode-extension has no producer in this repo — it ships in AltimateAI/vscode-altimate-mcp-server#453. Release this CLI first, so the reader exists before the extension starts writing .install-source; otherwise that file is orphaned by pre-#1096 readers and can be misattributed later. A zero vscode-extension share after release means the extension hasn't rolled out yet, not that there are no extension installs — now noted in telemetry.md.
Verification: typecheck clean, 1383 pass / 0 fail across test/cli test/install test/telemetry test/branding. The install.ps1 Pester additions are unverified locally (no pwsh on this machine) and rely on the CI job.
Round 3 (b69530bf0f) — both blocking items, plus four found pre-submit
Blocking, from review:
postinstall.mjswrite order — fixed. The previous commit message claimed "all three writers"; only two had been changed.- Pester test that could not fail — fixed.
$ErrorActionPreference = "Stop"inBeforeAll, a no-marker assertion proving the failure path was reached, and a third case asserting the preference itself so drift back toContinuefails visibly.
Found before submitting:
- Atomic trigger publish in all four writers. Companion-first ordering closes the "attribution lost" window but not truncation: a plain write truncates before filling, so a reader mid-write can observe an empty
.installed-version, whichwelcome.tsdeletes unread — losing the install itself. Only the extension published atomically. Now temp+rename ininstall,install.ps1andpostinstall.mjstoo. clearInstallSourcecould not do what its comment claimed. The comment said a directory in place of.install-sourceis still cleared;fs.unlinkSyncthrowsEPERM/EISDIRon a directory, so it would have survived every launch and pinnedinstall_methodtounknownpermanently. NowrmSyncwithforce+recursive, with a test that creates a directory-shaped file and asserts removal.- Pester header no longer described the file — read as subprocess-only after AST-extracted execution was added. Rewritten to document both layers.
- Empty-
USERPROFILEPester case wrote into the git checkout.[IO.Path]::Combine("", ".local", "share")returns the relative path.local\share, so the marker landed under the Pester process's cwd (checkout root in CI) whileAfterEachonly cleaned the sandbox. NowPush-Location $script:Sandbox. - Two test-quality fixes: the bash "unwritable dir" case asserted only
exit 0(passed with the writer deleted — bash twin of the Pester finding), and executed tests leaked temp dirs on assertion failure. - Docs:
unknownadded to the FAQ'sinstall_methodlist;telemetry.mdnow states both the boolean schema and the string KQL form ofis_upgradeinstead of one ambiguously.
Verified: typecheck clean; 1388 pass / 0 fail across test/cli test/install test/telemetry test/branding, two consecutive runs; bash marker writer executed directly for both install methods with no temp residue.
Not verified locally: the install.ps1 and Pester changes. No pwsh on the dev machine (cask needs interactive sudo), so the Windows Installer (Pester) job is their only check — and a passing job confirms the tests pass, not that they can fail. The mutation test (delete the try/catch, confirm the case fails) has not been performed.