claudeloop-testing
Explains claudeloop's testing philosophy and pytest layout — fakes over mocks for every port, FakeClock/FakeSleeper for simulating multi-day waits without real sleeping, mandatory Hypothesis property tests for numeric/time-based invariants, per-layer coverage gates (100% for domain and application), and the documented
- License 许可
- MIT
- 最近更新
- 2026-11-15
- Stars
- ★ 5
- Forks
- 3
01 前言 Front Matter
- name
- claudeloop-testing
- description
- Explains claudeloop's testing philosophy and pytest layout — fakes over mocks for every port, FakeClock/FakeSleeper for simulating multi-day waits without real sleeping, mandatory Hypothesis property tests for numeric/time-based invariants, per-layer coverage gates (100% for domain and application), and the documented
03 SKILL.md 正文
claudeloop testing philosophy
Codex skill mirror of
.claude/skills/claudeloop-testing/SKILL.md. When this guidance changes, update Claude skill, Cursor rule, and.agents/skills/in the same PR.
Layout
tests/domain/ # pure unit + Hypothesis property tests
tests/application/ # fakes for every port (fakes.py), zero real I/O
tests/infrastructure/ # adapters (incl. scripted test-agent unit tests)
tests/cli/
tests/live/ # opt-in free + paid live tiers
tests/live/system/ # marker: system — real FS/git/CLI + scripted agent
Run: pytest (skips live and system via addopts).
System harness: pytest -m system.
Live free: pytest -m live. Paid: pytest -m "live and paid" --run-paid-live.
See docs/guides/live-testing.md.
Coverage: pytest --cov-report=term-missing (already the default via
addopts in pyproject.toml).
Test-only agent gate (composition root only; not a user feature):
CLAUDELOOP_ALLOW_TEST_AGENT=1 + CLAUDELOOP_TEST_AGENT_SCRIPT=<json>.
Coverage is per-layer, not global — and why
There is deliberately no blanket --cov-fail-under in pyproject.toml.
CI enforces coverage separately per layer: 100% on domain/ and
application/ (zero I/O, zero third-party deps — no excuse for an untested
branch, and this is the code deciding whether an unattended multi-day run
keeps going, waits, or gives up), a lower floor on infrastructure//cli/.
When adding domain code, run coverage scoped to it and treat anything less
than 100% as a defect, not a number to negotiate down.
Fakes, never unittest.mock.Mock, for ports
Every port in application/ports.py gets a real class
implementing the same Protocol — checked by mypy --strict against the
port shape. A Mock has no such check; a port method rename silently
breaks nothing in test code, possibly not even at runtime. If you're
tempted to from unittest.mock import Mock for a port, write a small
FakeXxx class instead.
FakeClock / FakeSleeper — never sleep for real in a test
domain/waiting.py is designed entirely around instants
(next_probe_instant() -> datetime), never durations, specifically so
tests never call real time.sleep(). Pattern (tests/application/fakes.py):
clock = FakeClock(start=NOW)
sleeper = FakeSleeper(clock) # sleep_until(instant) jumps clock straight there, no real delay
This is what lets a test simulate a seven-day rate-limit wait or the
credit-top-up scenario (already covered at the domain layer —
test_credit_topup_sequence_resumes_after_several_failed_probes in
tests/domain/test_loop.py) in milliseconds. Do not use
unittest.mock.patch("time.sleep") — a real fake port is one thing to
reason about; a patched stdlib call is something every test touching timing
has to remember to apply consistently.
Hypothesis property tests are mandatory for numeric/time-based logic
Not a nice-to-have. test_property_credits_probe_never_in_the_past_and_never_exceeds_ceiling
in tests/domain/test_waiting.py caught a real production-shaped bug during
development: probe_count=29, factor=3.0 overflowed timedelta's max
magnitude via unclamped exponential backoff — an example-based test at any
hand-picked probe_count would never have found this. When you add a
numeric config field (an interval, ceiling, factor, budget cap), write a
@given(...) test asserting the invariant holds across the entire valid
input space, not a handful of examples. Two existing property tests to use
as templates: the one above, and
test_property_never_proposes_instant_beyond_max_wait.
Golden fixtures from real transcripts
Prefer capturing real observed SDK/API payloads as test fixtures over
inventing synthetic ones — a real credits_required 429 rejection from a
development transcript is already used this way; the shape of a real error
payload is rarely exactly what you'd guess.
# pragma: no cover — reserved, and always justified inline
Every use must carry a comment explaining why the branch is unreachable, not just that it is. Two real examples already in the codebase:
if candidate < now: # pragma: no cover — unreachable: all config intervals are
candidate = now # validated positive in __post_init__, so every branch above
# already yields candidate >= now. Kept as a defensive invariant guard.
A bare # pragma: no cover with no reasoning will be rejected in review —
grep the codebase for the pattern above before adding a new one, and match
that level of specificity.
Full reference
docs/contributing/testing.md.
04 同仓库更多 Skills
white-paper-writing
Use when writing white papers, technical reports, policy documents, thought leadership pieces, or any long-form authoritative document. Triggers on requests to write, structure, outline, or review white papers, technical reports, position papers, policy briefs, or evidence-driven arguments. Also use for B2B content marketing strategy, technical documentation requiring authority, academic-style reports, or any document that must educate while persuading. Use when someone asks "how should I structure this" for a long-form document.
prose-technical-mechanics
Use when editing or writing prose at a technical/mechanical level: choosing between Anglo-Saxon and Latinate diction, placing information in topic vs stress positions, writing periodic vs cumulative sentences, handling sentence rhythm, punctuation as logic, transitions, paragraph structure, or phonesthetics. Use when someone wants to understand WHY prose works or doesn't, not just surface-level style advice. Triggers on word choice, sentence structure, emphasis, rhythm, diction, or any deep writing mechanics question.
prose-mechanics
Use when writing, editing, or analyzing prose style; when trying to avoid generic AI-sounding writing; when applying specific prose styles such as journalistic, academic, narrative, KJV biblical, Hemingway plain, Latinate/Jamesian, Orwellian, Augustan, Victorian, Early Modern (Bacon, Browne, Milton), or free indirect discourse; or when coaching someone on writing voice and mechanics. Triggers on any request to improve, analyze, or write in a specific prose style, to fix "AI-sounding" writing, to make prose more vivid or concrete, to vary sentence rhythm, or to understand how a particular writer achieves their effects.
narrative-structure
Use when structuring stories, pitches, presentations, white papers, or any persuasive narrative; analyzing the hero's journey in fiction or non-fiction; applying StoryBrand for business communication; designing narrative arcs for thought leadership; or understanding Vogler, Snyder, Propp, or Campbell frameworks. Triggers on any narrative structure, storytelling, pitch structure, StoryBrand, hero's journey, three-act structure, Freytag, Save the Cat, narrative arc, or "how should I structure this story/pitch/presentation" request.
legal-brief-writing
Use when writing legal briefs, motions, or any court filing; when analyzing judicial cognition and persuasion; when applying elite brief-writing techniques from Roberts, Clement, Kagan, Garner; or when structuring legal arguments for maximum persuasion. Also use for autistic/neurodivergent advocates who need to understand how their natural communication style maps to elite legal advocacy. Triggers on any legal writing, brief drafting, motion preparation, or legal argument structuring request.
journal-publishability-criteria
Use whenever a research paper, article or preprint is being written, revised, reviewed or published, and specifically before every publication of docs/paper.md. Covers what legitimate journals actually gate on, in order of gating — the editor's desk screen (scope, novelty, package problems), what reviewers judge (methodological soundness, claims the evidence can carry, statistical and qualitative reporting), EQUATOR reporting guidelines, the ICMJE authorship criteria, COPE's view of text recycling, the near-universal generative-AI policy (no AI author, disclosure, author responsibility), TOP 2025 transparency standards, how to verify a journal is legitimate, submitting without an institution, and a pre-submission checklist. Triggers on requests to evaluate, review, prepare or submit a paper, to judge publishability, to write a declarations or data-availability section, to disclose AI use, or to check a journal.