testing
Use when designing, writing, or reviewing tests — deciding what to test, structuring a suite, adding randomized or exhaustive tests, making failures reproducible, or dealing with slow or flaky tests
- License 许可
- MIT
- 最近更新
- 2026-11-14
- Stars
- ★ 10
- Forks
- 0
01 前言 Front Matter
- name
- testing
- description
- Use when designing, writing, or reviewing tests — deciding what to test, structuring a suite, adding randomized or exhaustive tests, making failures reproducible, or dealing with slow or flaky tests
03 SKILL.md 正文
Testing
How I design test suites. The defaults are what I follow; the last section lists ideas to weigh case by case. Points drawn from elsewhere link their source.
This is a self-improving skill. If you used it and it came up short — a missing command, flag, gotcha, or workflow — invoke the self-improving-skills skill and follow it before you finish.
Defaults
Test features at the boundary
A suite exists to make change cheap. Test what the software does at its boundary — the public API for a library, what the user sees for an app — not its internal functions. If rewriting the implementation would break the tests, they pin code, not features. (How to Test)
Substitute explicitly
When a test needs a stand-in, inject it as a dependency, or subclass the real type with one that honors its contract (Liskov substitution). Never monkeypatch.
Optimize purity, not size
Classify tests by purity (how much IO, threading, and timing they depend on) and extent (how much code they exercise), not "unit" versus "integration." Each step toward pure, single-threaded computation buys about half an order of magnitude of speed and removes a source of flakes, so push hard on purity. Let extent be whatever the feature needs. (Unit and Integration Tests)
Get purity through sans-IO design: the caller does IO and passes values in, and the core computes and returns values.
Make new cases data
Shape the code under test as value in, value out, so a new case is data — an
inline string or a file under test_data/ — not a new test function. Adding
a test must be trivial, or people stop adding them.
Generate inputs before writing examples
Reach for a randomized or property test first, and fall back to examples or
snapshots when inputs can't be generated. Keep a couple of example tests as
smoke checks. Write property tests with the hegel-skill:hegel skill.
- Bugs live in small, tricky inputs. When the domain allows it, enumerate every input up to a bound instead of sampling: a generator that takes an RNG becomes an enumerator when the RNG is rigged to emit every sequence. (Generate All the Things)
- Pass the RNG explicitly; never use an ambient global. Every randomized failure prints a seed that reproduces it. (On Random Numbers)
- When a fuzzer misses a bug found some other way, fix the fuzzer until it finds the bug, then fix the code. (Finding Bugs)
Keep main green
Merge through a queue that tests the merge result, so main always passes. Then every failure on main is a flake: keep a list of them and fix the most frequent first. (Catch Flakes On Main)
Keep the suite fast
Gate slow tests behind a runtime env var such as RUN_SLOW_TESTS, skipped
locally and run in CI — not behind conditional compilation. Print each
test's duration so slowness stays visible.
Worth weighing
Valid, but not defaults:
- Route every test through one
check(input, expected)helper, so an API change touches one place. (How to Test) - Prefer snapshot tests, where the framework rewrites the expected literal in update mode, over hand-written expectations. (Underusing Snapshot Testing, Snapshot Testing For the Masses)
- Compare against an oracle: a naive implementation, a model, or a second implementation. (Finding Bugs)
- Swarm testing: randomize operation weights per run so rare combinations happen. (Swarm Testing Data Structures)
- Coverage marks tie a branch to the test that exercises it, which keeps negative tests honest. (A Trick For Test Maintenance)
- Treat any repository check — lint, licenses, binary size — as a test, and ratchet improvements against a list of known violators. (Of Rats and Ratchets)
- Deeper techniques: finite-entropy PRNGs that shrink for free (Test Case Minimization, Random Fuzzy Thoughts), deterministic thread interleavings (Properly Testing Concurrent Data Structures), liveness simulation (Simulation Testing For Liveness), and fuzzers at several levels of realism (A Tale Of Four Fuzzers).
- Rust: one integration-test crate and
doctest = falsein large workspaces. (Delete Cargo Integration Tests)
04 同仓库更多 Skills
zoekt
Use when searching code across repositories — prefer zoekt over grep/ripgrep for indexed code search. Covers indexing repos, search syntax, and query patterns.
writing-for-accessibility
Use when writing documentation, alt text, diagrams, or any content that should be accessible to people with disabilities. Covers alt text, color contrast, inclusive language, and accessible visuals. Also use when reviewing documents for accessibility.
writing-error-messages
Use when writing, reviewing, or improving error messages in code, CLI tools, APIs, or UIs. Covers identifying the cause, explaining the fix, and setting the right tone. Also use when error messages are vague, unactionable, or confusing.
working-in-the-open
Use when making design decisions during implementation — routes rationale to the right venue (commits, task comments, code comments, or documentation) based on how durable the context needs to be
web-standards
Use when writing or reviewing HTML, CSS, or browser JavaScript — covers semantic markup, native platform features over libraries, Baseline browser support, accessibility and APCA contrast, and optional validation with vnu, axe, and Lighthouse
web-design
Use when designing or building web UI, frontends, or styling — covers personal web design preferences, fluid layout, and Alpine.js