Test basecamp connect end to end against the fake Basecamp

RO
21d ago
+1106-0

PR 2 of a stack of 8 that tests basecamp connect end to end. Stacked on #822 (the fake Basecamp). It adds a test package and changes no behaviour of the CLI. It does not touch the fake.

Why

#822 gives us a Basecamp that speaks both lanes of the event feed. Nothing yet runs the real binary against it. The unit tests prove the parts. They do not prove that a person who runs auth agent connect, connect setup and connect gets one request line on stdout when the operator mentions the agent.

The failure scenarios that come next (PR 3), the grace knob (PR 5) and the dev BC3 target (PR 6) all need the same harness: a real basecamp connect process, its two outputs read as they are written, and waits that never sleep. This PR builds that harness and holds it to the happy path.

What

e2e/connect, a Go test package (//go:build linux || darwin, the platforms the connector runs on). It is inside go test ./..., so make test, bin/ci, the unit test job and the race shards all run it. It has no -short skip.

  • The binary. TestMain's run builds basecamp once, the first time a test needs it, so go test -list builds nothing. It is built the way the test binary was: -race when the test binary is a race build, and -tags dev when it is a dev build. Small race_tag / dev_tag test files set the constants, as internal/commands/dev_tag.go does. If a subprocess reports WARNING: DATA RACE, the test fails.
  • A home per test. NewHarness starts the fake and builds the CLI's whole environment from nothing: HOME, XDG_CONFIG_HOME, XDG_STATE_HOME, XDG_CACHE_HOME, BASECAMP_NO_KEYRING=1, BASECAMP_NO_UPDATE_CHECK=1, and BASECAMP_BASE_URL / BASECAMP_OAUTH_ISSUER at the fake. Only PATH, TMPDIR and GORACE pass through. So no BASECAMP_TOKEN, BASECAMP_NONINTERACTIVE or real config from the host can get in.
  • Setup through the CLI. Setup runs basecamp auth agent connect -P agent --no-browser, then basecamp connect setup -P agent --operator <id> --serve <id> --json. Run / MustRun run any one-shot command.
  • The connector. Connect (or Start for any command, in any directory) starts the process. Goroutines read stdout and stderr to the end the whole time, so the process never blocks on a full pipe. Stdout is parsed into typed lines: a pointer (no type), an event verdict or a request. The wire types are written out in the test rather than borrowed from the connector, so a protocol change shows up here. Stderr is parsed as slog records.
  • Waits. WaitFor(predicate) checks again each time the process writes, never on a timer. A 30s deadline only bounds a failure. If the process exits before the condition is true, the test fails at once with the transcript. WaitRunning and WaitStreaming(n) wait for connector: running and for the nth time the feed walk reaches its head. Interrupt sends SIGINT only after connector: running. Signal, Kill, Wait (exit code) and Stop (SIGINT, then exit 130) are also there.
  • Every run. At cleanup the process is killed if it is still running and then waited for. Every stdout line must be a protocol line, and no race can be reported. A failed test logs the full transcript.

Tests

  • Setup through the CLI writes connect.json with mode 0600, with the agent, the operator trust and the served project. The fake saw one connection ceremony.
  • A trusted mention gives one pointer line (lane live), one admitted event line and exactly one request line. The request line has the event id, comment.created, trigger mentioned, and the recording with its URL and project name. It also has reply_to (the parent message), the requester id and name, acknowledge: true, and the content with the agent's mention replaced by a space.
  • The same event on both lanes: it is emitted on Live and Poll, then the cable is dropped. The reconnect's catch-up walks the poll lane from before the event, and the test asserts that the fake served that walk. The result is still one pointer line, one event line and one request line. A later live mention makes sure the walk's output was all handled before the connector stops.
  • An untrusted mention (a project member who is not the operator) gives an event line discarded / untrusted_performer and no request line.
  • Stdout and stderr: every stdout line is valid JSON with a known shape. Stderr is all slog records, and it includes connector: running with the profile, account and agent.
  • The line and slog parsers have self-tests. They show that the stdout check can fail (plain text, null, arrays, unknown types, two objects on a line, a torn line).

The five scenarios run in parallel in about 3s (about 7s under -race). go test -race -count=10 ./e2e/connect passes, and also passes when it runs at the same time as -count=30 without race. bin/ci is green.

Summary by cubic

Adds an end-to-end test package that runs the real basecamp connect binary against the fake Basecamp, covering the setup and connector flow a person runs.

  • No CLI behavior changes; only a new test package, built on the fake from #822.
  • Tests run on Linux and macOS, in the regular test suite, with no -short skip.
  • The binary is built once per run, with the same flags as the test binary (-race, -tags dev), so a race in the connector fails its test.
  • Each test gets an isolated home environment so host config, tokens, and profiles can't leak in.
  • Waits are driven by process output or fake server state, with a deadline that only bounds a failure.

Coverage

  • Setup through the CLI (auth agent connect, connect setup) writes owner-only connect.json.
  • A trusted mention produces one pointer, one admitted event, and exactly one request line with the expected fields.
  • An event on both lanes is handed off once, including across a reconnect's catch-up walk.
  • An untrusted mention produces a discarded event line and no request.
  • Every stdout line must be valid NDJSON; stderr must be slog records.
  • The stdout and stderr parsers have self-tests, including refusal cases.