feat(free): free hosted model, no signup

Closed
AN
7d ago
+6437-136

Issue for this PR

Closes #1114

Type of change

  • New feature

What does this PR do?

Adds a free hosted model to altimate-code — no signup, no API key. It is the client half; the gateway that serves it lives in AltimateAI/altimate-gateway.

How it works. The user picks "(Free)" in the model picker and gets a disclosure dialog (default No). Only on Yes does the client mint a random install secret, send sha256(secret) — never the secret — to the gateway's /register, and store the short-lived key it gets back. The provider loader is read-only: it reads a stored credential or does nothing. It never registers, so nothing leaves the machine before consent. Keys rotate silently on a 401.

Not a clone of the existing provider template, in three ways that matter:

  • Project config cannot touch this provider. A repo-local provider["altimate-free"] entry is dropped where config is read, so every consumer inherits the denial. This started as a guard on baseURL, and review reopened it three more times — through npm (which getSDK() imports and hands the stored key to), through variants via a consumer that indexed config directly, and through a ModelsDev registry record winning the provider registration. Guarding the named field was never the fix.
  • Registration needs a per-launch capability the TUI holds, so the local server route cannot mint an identity on its own.
  • Credential writes are serialised and atomic. This one is not free-tier-specific: FSUtil.writeJson wrote then chmodded, so every provider's auth.json was briefly world-readable on creation, and making the write atomic without a shared lock turned a lost-update into a lost credential. Both Auth implementations now take one canonical-path lock and share one atomic writer.

Also here, and worth separating in review:

  • SystemPrompt.environment() (cwd, worktree, date) moved out of the head of the system prompt. It sat ahead of skills, instructions and ~99 tool schemas, and Vertex stops prefix-matching at the first differing byte. Measured cross-user: 6,142 of ~121,000 tokens cached before, and the shared prefix is bounded by tool declarations serialising after systemInstruction — so this is worth ~1.18x, not the 9.6x a byte-identical fixture suggests. It applies to all Gemini traffic, including users on their own keys.
  • Deterministic MCP tool and skill ordering. MCP tools were assigned in connection-completion order and skills sorted with localeCompare, whose default follows the runtime's LANG/ICU data — so two machines emitted different bytes and shared no prefix. Skills needed fixing in two places; correcting either alone does nothing.
  • The two free-tier 429s get distinct messages, branched on the body discriminator rather than the status code, because budget statuses are inconsistent across LiteLLM versions. budget_exceeded is marked non-retryable.

How did you verify your code works?

Against the real gateway, real Vertex and our Langfuse, not mocks: script/e2e-free-tier.sh — 25 assertions, run live. Consent → registration → a real gemini-2.5-flash completion → the trace in Langfuse with a server-derived userId, a per-principal namespaced session, and a planted AWS key stored as [REDACTED:aws_access_key]. A recording proxy proves the negative: an install that has not consented sends the gateway nothing at all. auth.json lands at 0600 and the whole request log is scanned for the raw install secret. --dry-run rehearses at no cost and FAKE_BREAK= fault-injection shows each assertion still goes red for its own reason.

Four rounds of adversarial review, each returning findings, all fixed. Every test added here was verified by reverting its fix and confirming the test goes red — seven tests in this work passed against the bug they targeted before that check was applied, including a file-mode assertion that passed against the very non-atomic writer it existed to catch (the discriminator is the inode).

Checks: bun turbo typecheck --force 13/13, marker check clean across 15 upstream-shared files, and the affected suites green. Pre-existing and not from this branch: 26 core failures, 9 TUI, 5 in test/mcp/headers.test.ts (flaky) — confirmed identical at main.

Not verified: the TUI keystroke path is covered by component tests rather than driven headlessly in the live run, so "consent gates the network call" is proven from both sides but not in one continuous keyboard-to-Langfuse run. 18 of 39 changed files fail prettier; that is pre-existing, no CI or hook enforces it, and none of the added lines are affected.

Screenshots / recordings

No UI screenshots — the dialog is covered by component tests in packages/tui/test/cli/tui/dialog-free-gemini.test.tsx.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Summary by CodeRabbit

  • New Features
    • Added optional Gemini Flash (Free) access through the Altimate gateway.
    • Added consent disclosure, no-signup onboarding, registration, credential rotation, and provider status updates.
    • Added clearer messaging for rate limits and oversized requests.
  • Bug Fixes
    • Improved credential-store safety during concurrent updates and filesystem failures.
    • Made skill, tool, and prompt ordering consistent across locales and environments.
    • Improved lock cleanup error reporting and atomic file writes.
  • Documentation
    • Documented free-tier setup, telemetry events, usage limits, key rotation, and configuration options.

[!NOTE]
High Risk
Touches authentication storage, credential registration, and provider routing with new network trust boundaries; incorrect locking or config bypass could leak or corrupt API keys.

Overview
Introduces Gemini Flash (Free) as an altimate-free provider: users accept a disclosure in the model picker, then the CLI registers with the gateway using a separate install secret (hashed on the wire), stores a short-lived key, and loads the model read-only until credentials exist. Registration is gated by a per-launch consent token on POST /altimate/free/register, and inference uses authorizedFetch with origin checks and bounded 401 recovery.

Security and credentials: Project config and registry overrides for altimate-free are stripped at ingest so repo-local config cannot redirect keys or npm modules. auth.json writes go through a shared atomic writer, canonical-path locking across both Auth implementations, a unified schema preserving metadata, and stricter mutation reads so a failed read cannot wipe the whole store.

Reliability and UX: Free-tier 429/413 responses get distinct, mostly non-retryable user messages. Onboarding telemetry adds altimate_free and free-Gemini funnel events. Docs rename the managed gateway to altimate-backend and document the free tier and its telemetry identifier.

Caching (all providers): System prompt assembly moves volatile <env> later; skills/MCP tools sort by Unicode code point for stable prefixes. Altimate gateway docs use altimate-backend/altimate-default.

Reviewed by Cursor Bugbot for commit 810dc11fcbdb18467f437f178198fc349010771d. Bugbot is set up for automated code reviews on this repo. Configure here.