clirunner-env
Use when a CLI test builds env= as a filtered-dict, passes env={}, or relies on monkeypatch.delenv. Click's CliRunner.invoke only overrides keys the dict names; pass {key: None} to delete one.
- License 许可
- MIT
- 最近更新
- 2026-11-15
- Stars
- ★ 1
- Forks
- 1
01 前言 Front Matter
- name
- clirunner-env
- description
- Use when a CLI test builds env= as a filtered-dict, passes env={}, or relies on monkeypatch.delenv. Click's CliRunner.invoke only overrides keys the dict names; pass {key: None} to delete one.
- disable-model-invocation
- true
02 资源文件 3
- 参考文档evals/evals.json3.0 KB
- 参考文档EVIDENCE.md5.3 KB
- 参考文档gotchas.md2.3 KB
03 SKILL.md 正文
Click CliRunner: env=None deletes; absence does NOT
Problem
You wrote a CLI test where the SUT (system under test) branches on whether an env var is set. The test invokes the CLI via CliRunner.invoke(cli, args, env=clean_env_dict), where clean_env_dict omits the env var you want absent. The test passes locally. You ship it. Later it turns out the test was never exercising the "var absent" branch — it was running with the var present (inherited from your shell), and your assertion happened to also pass for the "var present" code path. In a live-API-call test, this can mean the test SILENTLY makes the network call you thought you were preventing.
Root cause
Click's CliRunner.invoke(env=...) is an OVERRIDE dict, not a REPLACEMENT environment. It iterates the dict and applies each key:
env[key] = "value"→ setsos.environ[key] = "value"for the duration of the invocation, restoring afterwardenv[key] = None→ callsdel os.environ[key]for the duration of the invocation, restoring afterward- key absent from
env→os.environ[key]is left UNTOUCHED
Originally verified against click/testing.py:534 on Click 8.1.x; re-checked 2026-08-23 and 2026-08-24 against the current published testing module source; re-checked 2026-10-06 against Click 8.5.x (the stable docs header and the published testing module). The durable evidence is the signature: env: Mapping[str, str | None] | None on CliRunner, CliRunner.invoke and CliRunner.isolation. A value type of str | None is the API stating that None is a meaningful value rather than an omission — that is the delete. The docs describe env as "overrides", which is the absent-keys-untouched half. Cite the signature, not a file offset or a version pin — the 8.1.x branch no longer exists, and the signature has survived every check so far.
Trigger conditions
You wrote one of these patterns and the test silently passed:
# Pattern 1: filtered dict
clean_env = {k: v for k, v in os.environ.items() if k != "ANTHROPIC_API_KEY"}
runner.invoke(cli, args, env=clean_env) # ANTHROPIC_API_KEY may STILL be in os.environ during invoke
# Pattern 2: empty dict
runner.invoke(cli, args, env={}) # NOTHING is deleted; nothing is overridden; identity to no env=
# Pattern 3: subset dict
runner.invoke(cli, args, env={"OTHER_VAR": "x"}) # Only OTHER_VAR is touched
Specific symptoms:
- Test passes when run alone but you can't reproduce the bug it was supposed to catch
os.environ.get("MY_VAR")inside the SUT returns a non-empty string even though the test "set"envto a dict withoutMY_VAR- A pre-flight key check passes when you expected it to fail
- A test that asserts "refuse when key absent" never actually fires its refusal path
Fix
Pass {key: None} explicitly for every env var that must be absent:
runner.invoke(cli, args, env={
"ANTHROPIC_API_KEY": None, # delete for the duration of invoke
"OPENROUTER_API_KEY": None, # delete for the duration of invoke
"SOME_OVERRIDE": "value-x", # set for the duration of invoke
})
For pytest-based tests, prefer monkeypatch.delenv when the code under test is not exclusively a CLI entry-point — the SUT then runs with the key genuinely absent from os.environ, which is the clearer semantics:
def test_resolver_no_keys(monkeypatch):
monkeypatch.delenv("ANTHROPIC_API_KEY", raising=False)
monkeypatch.delenv("OPENROUTER_API_KEY", raising=False)
# ... assertion
When you MUST use CliRunner's env= (the SUT is launched as a click command and you need one invocation), use the {key: None} form.
Verification
After the fix, verify the test actually exercises the intended branch:
- Temporarily insert
print(f"key={os.environ.get('MY_VAR')!r}", file=sys.stderr)inside the SUT and re-run. The print should showkey=None. - Run the test on a machine where the env var IS set in the shell. A correct fix passes/fails the same way as on a clean machine.
- If the test asserts an exception, intentionally remove the
{key: None}and confirm the test then fails. Put it back.
Notes
Open gotchas.md when a CLI test silently passed on a branch that never ran the absent-env path. It records the origin incident and how this card's own references rotted.
- Most dangerous in tests of API-key-dependent code: the missing
{key: None}can let a test silently make a live network call, burning real money and producing flaky results. - The behavior is documented — the env parameter on CliRunner.invoke — but phrased as "added/overridden," not as "absence does nothing." Easy to miss.
subprocess.run(env=...)works differently: there,env=REPLACES the entire environment. The cross-tool inconsistency is a contributor to the confusion.
Example (2026-06-09)
# tests/ablation/test_cli_d3_fixes.py — BEFORE, looked correct
clean_env = {k: v for k, v in os.environ.items() if k != "ANTHROPIC_API_KEY"}
result = runner.invoke(run_ablation, [...], env=clean_env)
assert "ANTHROPIC_API_KEY is not set" in result.output
On a machine where OPENROUTER_API_KEY was set, a model-aware resolver in the SUT rewrote the model id and proceeded to a live 12-minute Anthropic API call. The test passed because the old assertion string was still in result.output from a DIFFERENT code path — the branch under test was never exercised.
# AFTER
result = runner.invoke(run_ablation, [...], env={
"ANTHROPIC_API_KEY": None,
"OPENROUTER_API_KEY": None,
})
assert "ANTHROPIC_API_KEY is not set" in result.output # now actually fires
04 同仓库更多 Skills
subagent-handback
Use when prompting, briefing or dispatching a research subagent, or when one goes idle with no report. Name a return channel, check its tool grant, verify every claim, negatives first.
disposition-schema
Use when dispatching parallel agents to verify or adjudicate findings (adversarial verify, security disposition, council fires), or before you consolidate their outputs and risk an ID collision.
decision-rights
Use when writing a handoff, plan, ADR or subagent brief, or one says "do not re-litigate" or "final". Separates evidence access from decision rights and marks what is non-negotiable.
dead-predicate
Use when a hook didn't fire, a reminder never arrived, or before claiming a rule is hook-enforced: a live router rule can match nothing (missing word, JSON escape, shadowing, stale name).
vacuous-check
Fail-open success checks. Use when a script says OK but nothing happened, a probe batch returns a false negative or 0 results, or `String(got) === String(want)` passes on a type mismatch.
stale-deploy
Use when asking "is it live", a deploy looks stale, or a poll loop exits instantly. Verify a GitHub Pages or CDN deploy serves new content: poll a marker new to this deploy.