feat(evals): keep named login variables in claude.sh with EOR_KEEP_ENV

Merged
BO
3d ago
+77-2

evals/end-of-reply/models/claude.sh unsets every CLAUDE* and ANTHROPIC* variable. A run that authenticates through the environment (CLAUDE_CONFIG_DIR plus gateway token variables) then falls back to the default login and its weekly limit, or fails with "Not logged in".

This adds an opt-in keep list. EOR_KEEP_ENV is a space-separated list of variable names the wrapper leaves set. Names match whole, so ANTHROPIC does not keep ANTHROPIC_AUTH_TOKEN. With EOR_KEEP_ENV unset or empty, the wrapper unsets the same variables as before.

Changes:

  • claude.sh: the unset loop skips names in the keep list (4 lines), and the header comment names the variable.
  • evals/end-of-reply/README.md: three lines under the wrapper table saying what is unset and how to keep a login.
  • CHANGELOG.md: an Unreleased entry marked repository only.
  • test/eval-wrappers.test.mjs: runs claude.sh with a stub claude on PATH that prints the CLAUDE*/ANTHROPIC* names it can see. It covers the default (only the wrapper's own CLAUDE_CODE_DISABLE_AUTO_MEMORY is visible), a keep list, and whole-name matching. Skipped on Windows.

Proof I ran:

  • Against the old claude.sh, the keep-list test fails; the other two pass because they describe unchanged behavior. With the change, all 3 pass.
  • node --test test/*.test.mjs: 182 pass, 0 fail, 7 skipped (exit 0). bun test test/eval-wrappers.test.mjs: 3 pass.
  • npx --yes oxfmt@0.70.0 --check . exit 0; npx --yes oxlint@1.85.0 . exit 0 (one existing warning in test/install-ps1.test.mjs).
  • Local hr-code-review of the final diff found no defects; it led to two prose edits in the changelog entry.

Not done: this does not keep variables by prefix, and it does not change the default isolation of any other wrapper.

  • The checks in the README's Development section pass (Node tests, Bun test of the new file, pinned oxfmt and oxlint; check:package and bun test skills/ not run, no packaged or skill file changed).
  • A change users will notice has a CHANGELOG.md entry under Unreleased.
  • Nothing names a private project or repository, a local path, a home directory or a credential.

Summary by cubic

Adds EOR_KEEP_ENV to claude.sh so a run can keep named CLAUDE* or ANTHROPIC* variables, allowing logins that live in the environment. Previously the wrapper unset every such variable, so an environment-based login (config directory or gateway token) fell back to the default CLI login or failed. The default remains the same: with EOR_KEEP_ENV unset or empty, all those variables are still unset.

  • EOR_KEEP_ENV is a space-separated list; names match whole, so ANTHROPIC does not keep ANTHROPIC_AUTH_TOKEN.
  • Adds tests for the default, keep-list, and whole-name matching (skipped on Windows), plus a changelog entry and README notes.