fix(host-service): classify an unreachable cloud API as 503, not a 500

Merged
SA
3mo ago
+243-3

Problem

host.info is a small procedure the desktop polls: it reports which host this is, and which organization it belongs to. Resolving the organization means an outbound tRPC call to our cloud API, and that call's transport failures were unhandled. Any network problem between the user's machine and us escaped getOrganization as a raw TRPCClientError: fetch failed, became an INTERNAL_SERVER_ERROR, and — under the #6164 contract, where a 500 means "we have a bug" — got reported to Sentry as a host-service defect.

The captured causes are unambiguously the user's network, not our software:

  • HOST-SERVICE-1B (51 events, escalating) — Error: read ETIMEDOUT on an established socket.
  • HOST-SERVICE-43 — undici HeadersTimeoutError.
  • HOST-SERVICE-46 — a captive wifi portal terminated TLS and presented its own certificate: Hostname/IP does not match certificate's altnames, for a hostname that is not ours.

Why fix rather than archive again. Two earlier issues in this family were archived on 2026-08-19. They came back, because the Sentry grouping key follows the innermost transport error — so each new flavour of network failure from this one throw site opens a new issue. Archiving does not converge; it is unbounded manual triage. And the classification is wrong on the merits: someone's captive portal is not an internal server error.

Reach. The failing builds are 1.22.0 / 1.23.0 / 1.24.1; the package is at 1.24.1, so this ships in the next host-service release and reaches machines as the desktop app auto-updates and respawns its bundled host-service. The existing issues will keep firing from older builds until the fleet rolls forward.

Root cause

getOrganization awaited api.organization.getByIdFromJwt.query(...) with no adapter at the boundary. The one-hour success cache means this only fires when the call actually goes out.

Fix

One adapter, cloud-api-error.ts, applied at that single call site.

The producer is a tRPC client over Node's HTTP stack — we do not own it, so the error has to be reconstructed rather than recognised. The most structural signal available is the machine-readable code Node and undici attach to exactly these failures, which sits three levels down the cause chain: TRPCClientError("fetch failed") → TypeError("fetch failed") → the coded error. The adapter walks that chain structurally (duck-typed, never instanceof — the links are not ours) and matches code against a closed set. No message regexes.

Matched: ECONNREFUSED, ECONNRESET, ETIMEDOUT, ENOTFOUND, EAI_AGAIN, EHOSTUNREACH, ENETUNREACH, UND_ERR_CONNECT_TIMEOUT, UND_ERR_HEADERS_TIMEOUT, and the certificate-validation codes a TLS-intercepting portal or proxy produces: ERR_TLS_CERT_ALTNAME_INVALID, UNABLE_TO_VERIFY_LEAF_SIGNATURE, UNABLE_TO_GET_ISSUER_CERT, UNABLE_TO_GET_ISSUER_CERT_LOCALLY, SELF_SIGNED_CERT_IN_CHAIN, DEPTH_ZERO_SELF_SIGNED_CERT.

A match becomes SERVICE_UNAVAILABLE — already this package's idiom for "the thing you need to reach is not reachable" — carrying the underlying message, which is the informative one (fetch failed is not). Behaviour is otherwise untouched: same cache, same PRECONDITION_FAILED when the organization is genuinely missing, no retries or backoff added.

What deliberately still reports as a 500

  • HOST-SERVICE-41 — an HTML page where JSON was expected. It surfaces as a SyntaxError with no transport code attached, so the adapter does not match it and it stays a 500. This is intentional: at this layer a non-JSON response body is indistinguishable from our own cloud genuinely returning something broken, and silencing it would hide a real server-side bug. It is being archived separately.
  • Application errors the cloud API returned (auth failures, not-found). These arrive as a TRPCClientError built from an error response, with no transport code anywhere in the chain — they keep reporting.
  • Any failure with no cause chain at all.
  • CERT_HAS_EXPIRED / CERT_NOT_YET_VALID — excluded on purpose, and the one judgement call here. Interception announces itself as an untrusted issuer or the wrong hostname; an expired certificate is at least as likely to be ours, and if our certificate lapses this procedure should be one of the things that screams.

Worth naming the residual over-match: ECONNRESET / ECONNREFUSED could in principle be our own edge misbehaving rather than the user's network. From the host's vantage point that still is "could not reach the cloud", and cloud-side monitoring owns that failure — it is not something a desktop host-service should be diagnosing.

No typed cause

Thrown as a plain new TRPCError({ code, message }) with no cause: { kind }. Nothing reads one: the only desktop callers of host.info are useRemoteHostStatus, which branches on isSuccess alone and falls through to ready otherwise, and the tray menu, which does catch { return null }. A kind nobody reads is code maintained for nothing; it can be added when a renderer branch wants it.

Scope

Deliberately confined to this one procedure's outbound call. Other host-service procedures also call the cloud API — workspaces.ts:456 (task.byId), workspaces.ts:501 (v2Workspace.trackCreated), project.ts:453 (v2Project.findByGitHubRemote), plus fire-and-forget calls in workspace.ts, notifications.ts, and local-workspace-store.ts that already swallow their own rejections. They are left alone; each deserves its own look at what its caller should see. Classifying inside createApiClient's link chain would have covered them all in one move, and was rejected for exactly that reason — it would silently reclassify call sites nobody has reviewed.

Verification

The negative tests are the point of this change, so both directions were checked against a strawman first.

Strawman 1 — message regex on fetch failed (the last-resort approach this task rules out). Two negatives fail, and every positive fails too, because the client error's message is fetch failed and the real message is unreachable from there:

(fail) rethrowCloudUnreachable > read timeout on an established socket (HOST-SERVICE-1B)
(fail) rethrowCloudUnreachable > undici headers timeout (HOST-SERVICE-43)
(fail) rethrowCloudUnreachable > captive portal presents its own certificate (HOST-SERVICE-46)
... 12 more positives ...
(fail) rethrowCloudUnreachable > an error with no cause chain still reports
(fail) rethrowCloudUnreachable > an expired certificate still reports
 4 pass
 17 fail

Strawman 2 — blanket "anything this call throws is unreachable" (the tempting wrong fix). All five negatives fail:

(fail) rethrowCloudUnreachable > an HTML body where JSON was expected still reports (HOST-SERVICE-41)
(fail) rethrowCloudUnreachable > an application error from the cloud API still reports
(fail) rethrowCloudUnreachable > a not-found from the cloud API still reports
(fail) rethrowCloudUnreachable > an error with no cause chain still reports
(fail) rethrowCloudUnreachable > an expired certificate still reports
(fail) rethrowCloudUnreachable > our own typed error passes through untouched
 4 pass
 17 fail

With the shipped adapter — fixtures built with the real TRPCClientError and real captured shapes, not mocks:

$ bun test src/trpc/router/host
bun test v1.3.14 (0d9b296a)

 21 pass
 0 fail
 37 expect() calls
Ran 21 tests across 1 file. [34.00ms]

$ cd packages/host-service && bun run typecheck
$ tsc --noEmit --emitDeclarationOnly false

$ bunx biome check packages/host-service/src/trpc/router/host/host.ts \
    packages/host-service/src/trpc/router/host/cloud-api-error.ts \
    packages/host-service/src/trpc/router/host/cloud-api-error.test.ts
Checked 3 files in 5ms. No fixes applied.

(--write was run once for formatting, then re-checked clean as above. Repo-wide bun run lint was skipped per the known cross-worktree bun-lock hang.)

Refs HOST-SERVICE-1B
Refs HOST-SERVICE-46
Refs HOST-SERVICE-43

https://claude.ai/code/session_01MLQT5zzzaRZgrnU4Tbz27Y

Summary by cubic

Classifies cloud API transport failures in host.info as 503 Service Unavailable instead of 500 Internal Server Error, so Sentry stops flagging user network issues as host-service bugs. Previously, a TRPCClientError: fetch failed bubbled up as a 500.

  • Adds a small adapter that walks the TRPCClientError → TypeError → coded error cause chain and maps Node/undici transport codes (timeouts, DNS, connection resets/refusals, and untrusted TLS issuer/hostname) to SERVICE_UNAVAILABLE, preserving the underlying message; no regex.
  • Applies the adapter only at the host.info organization lookup call site; other host-service cloud calls are unchanged by design.
  • Keeps 500s for non-JSON responses, cloud application errors, unknown failures with no code, and expired/not-yet-valid certificates.
  • Behavior otherwise unchanged: same cache, no retries/backoff, no API surface changes. Includes focused tests for the adapter.
  • Rollout: ships with the next host-service release bundled in the desktop app; older builds will continue emitting 500s until auto-update advances.

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of cloud connectivity failures during organization lookup.
    • Network, DNS, timeout, connection, and certificate trust errors now report a clear service-unavailable message.
    • Application errors, malformed responses, and unrelated certificate errors continue to be reported normally.
  • Tests
    • Added coverage for supported unreachable-cloud scenarios and errors that should remain unchanged.