AGENT SKILL

mutation-testing

Validate test effectiveness with mutation testing using Stryker (TypeScript/JavaScript with Vitest or bun test via @hughescr/stryker-bun-runner) and mutmut (Python). Find weak tests that pass despite code mutations. Use to improve test quality.

License 许可
MIT
最近更新
2026-09-24
Stars
★ 0
Forks
0

01 前言 Front Matter

name
mutation-testing
license
MIT
description
Validate test effectiveness with mutation testing using Stryker (TypeScript/JavaScript with Vitest or bun test via @hughescr/stryker-bun-runner) and mutmut (Python). Find weak tests that pass despite code mutations. Use to improve test quality.
allowed-tools
Bash, Read, Edit, Write, Grep, Glob, TodoWrite

03 SKILL.md 正文

Mutation Testing

Expert knowledge for mutation testing - validating that your tests actually catch bugs by introducing deliberate code mutations.

Core Concept

  • Mutants: Small code changes introduced automatically
  • Killed: Test fails with mutation (good - test caught the bug)
  • Survived: Test passes with mutation (bad - weak test)
  • Score: Percentage of mutants killed (aim for 80%+)

TypeScript/JavaScript (Stryker)

Vitest Runner

Installation

# Using Bun
bun add -d @stryker-mutator/core @stryker-mutator/vitest-runner

# Using npm
npm install -D @stryker-mutator/core @stryker-mutator/vitest-runner

Configuration

// stryker.config.mjs
export default {
  packageManager: 'bun',
  reporters: ['html', 'clear-text', 'progress'],
  testRunner: 'vitest',
  coverageAnalysis: 'perTest',
  mutate: ['src/**/*.ts', '!src/**/*.test.ts'],
  thresholds: { high: 80, low: 60, break: 60 },
  incremental: true,
}

Bun Native Runner (bun test)

Use when projects use bun test directly (not Vitest).

Requirements

  • Bun >= 1.3.7 (needs TestReporter WebSocket events from Bun PR #25986)
  • @stryker-mutator/core ^9.0.0

Installation

bun add -D @hughescr/stryker-bun-runner @stryker-mutator/core

Configuration

// stryker.conf.mjs
export default {
  testRunner: 'bun',
  coverageAnalysis: 'perTest',
  mutate: ['src/**/*.ts', '!src/**/*.test.ts'],
  thresholds: { high: 80, low: 60, break: 60 },
  incremental: true,
  bun: {
    inspectorTimeout: 5000,  // WebSocket Inspector connection timeout (ms)
    // bunPath: '/path/to/bun',  // only if custom Bun install
    // timeout: 30000,           // test timeout (ms)
    // env: { DEBUG: 'true' },
    // bunArgs: ['--bail'],
  },
}

Key Behaviors

  • Sequential execution: Runs tests with --concurrency=1 for accurate per-test coverage. Slower than parallel but required for correct test-to-mutant correlation.
  • Concurrent test patching: Automatically patches describe.concurrent(), test.concurrent(), it.concurrent() to run sequentially during mutation testing — no code changes needed.
  • Inspector Protocol: Uses Bun's WebSocket Inspector API to discover tests and correlate coverage.

Running Stryker

# Run mutation testing
bunx stryker run

# Incremental mode (only changed files)
bunx stryker run --incremental

# Specific files
bunx stryker run --mutate "src/utils/**/*.ts"

# Open HTML report
open reports/mutation/html/index.html

Example: Weak Test

// Source code
function calculateDiscount(price: number, percentage: number): number {
  return price - (price * percentage / 100)
}

// ❌ WEAK: Test passes even if we mutate calculation
test('applies discount', () => {
  expect(calculateDiscount(100, 10)).toBeDefined() // Too weak!
})

// ✅ STRONG: Test catches mutation
test('applies discount correctly', () => {
  expect(calculateDiscount(100, 10)).toBe(90)
  expect(calculateDiscount(100, 20)).toBe(80)
  expect(calculateDiscount(50, 10)).toBe(45)
})

Python (mutmut)

Installation

uv add --dev mutmut

Running mutmut

# Run mutation testing
uv run mutmut run

# Show results
uv run mutmut results

# Show specific mutant
uv run mutmut show 1

# Generate HTML report
uv run mutmut html
open html/index.html

Common Mutation Types

// Arithmetic Operator
// Original: a + b → a - b, a * b, a / b

// Relational Operator
// Original: a > b → a >= b, a < b, a <= b

// Logical Operator
// Original: a && b → a || b

// Boolean Literal
// Original: true → false

Mutation Score Targets

Score Quality Action
90%+ Excellent Maintain quality
80-89% Good Small improvements
70-79% Acceptable Focus on weak areas
< 60% Poor Major improvements needed

Improving Weak Tests

Pattern: Insufficient Assertions

// Before: Mutation survives
test('calculates sum', () => {
  expect(sum([1, 2, 3])).toBeGreaterThan(0) // Weak!
})

// After: Mutation killed
test('calculates sum correctly', () => {
  expect(sum([1, 2, 3])).toBe(6)
  expect(sum([0, 0, 0])).toBe(0)
  expect(sum([])).toBe(0)
})

Pattern: Boundary Conditions

// After: Tests boundaries
test('validates age boundaries', () => {
  expect(isValidAge(18)).toBe(true)   // Min valid
  expect(isValidAge(17)).toBe(false)  // Just below
  expect(isValidAge(100)).toBe(true)  // Max valid
  expect(isValidAge(101)).toBe(false) // Just above
})

Best Practices

  • Start with core business logic modules
  • Ensure 80%+ coverage before mutation testing
  • Run incrementally (only changed files)
  • Focus on important files first
  • Don't expect 100% mutation score (equivalent mutants exist)

Workflow

# 1. Ensure good coverage first
bun test --coverage
# Target: 80%+ coverage

# 2. Run mutation testing
bunx stryker run

# 3. Check report
open reports/mutation/html/index.html

# 4. Fix survived mutants
# 5. Re-run incrementally
bunx stryker run --incremental
# or: npx stryker run --incremental

See Also

  • vitest-testing - Unit testing framework
  • test-quality-analysis - Detecting test smells

04 同仓库更多 Skills

aibizmod
aibizmod/aibizmod-service-web

{TODO=lowercase-hyphen-case-name}

[TODO: Write comprehensive description in third-person. Start with "This skill provides..." or "This skill should be used when..."] [TODO: Add "Use when" scenarios - specific situations where Claude should use this skill] [TODO: Add keywords - technologies, use cases, error messages that should trigger this skill]

★ 0
aibizmod
aibizmod/aibizmod-service-web

zustand-state-management

Zustand state management for React with TypeScript. Use for global state, Redux/Context API migration, localStorage persistence, slices pattern, devtools, Next.js SSR, or encountering hydration errors, TypeScript inference issues, persist middleware problems, infinite render loops.

★ 0
aibizmod
aibizmod/aibizmod-service-web

zod

TypeScript-first schema validation and type inference. Use for validating API requests/responses, form data, env vars, configs, defining type-safe schemas with runtime validation, transforming data, generating JSON Schema for OpenAPI/AI, or encountering missing validation errors, type inference issues, validation error handling problems. Zero dependencies (2kb gzipped).

★ 0
aibizmod
aibizmod/aibizmod-service-web

xss-prevention

XSS attack prevention with input sanitization, output encoding, Content Security Policy. Use for user-generated content, rich text editors, web application security, or encountering stored XSS, reflected XSS, DOM manipulation, script injection errors.

★ 0
aibizmod
aibizmod/aibizmod-service-web

wordpress-plugin-core

WordPress plugin development with hooks, security, REST API, custom post types. Use for plugin creation, $wpdb queries, Settings API, or encountering SQL injection, XSS, CSRF, nonce errors.

★ 0
aibizmod
aibizmod/aibizmod-service-web

woocommerce-dev-cycle

Run tests, linting, and quality checks for WooCommerce development. Use when running tests, fixing code style, or following the development workflow in WooCommerce projects.

★ 0