AGENT SKILL

jepsen-fault-testing

暂无简介

License 许可
Apache-2.0
最近更新
2026-11-14
Stars
★ 13
Forks
2

03 SKILL.md 正文

/jepsen-fault-testing

When to Use

  • Verify that distributed agent consensus (Raft, etcd) actually survives network splits
  • Prove that the distributed mutex never grants two agents the lock simultaneously
  • Chaos test before a production deployment of a multi-node agent cluster
  • Validate that CRDT merges produce linearizable results after partitions

Do NOT use for

  • Unit tests of business logic (use Jest [[jest-test-generation]])
  • Performance benchmarking (Jepsen introduces intentional faults, not load)

Jepsen test anatomy (Clojure DSL — conceptual)

;; A Jepsen test has 5 components:
{:name    "yamtam-raft-lock"
 :db      (raft-cluster-db)         ; setup/teardown cluster
 :client  (lock-client)             ; operations: acquire/release
 :nemesis (nemesis/partition-random-halves)  ; fault injector
 :checker (checker/linearizable)    ; correctness verifier
 :generator
   (->> (gen/mix [acquire-op release-op])
        (gen/stagger 0.1)           ; 100ms between ops
        (gen/nemesis
          (gen/seq [{:type :info :f :start}  ; inject partition
                    (gen/sleep 10)
                    {:type :info :f :stop}])) ; heal
        (gen/time-limit 60))}       ; 60s test

Node.js fault injection patterns (without Jepsen)

// Simulate network partition: drop packets between agent-1 and agent-2
// Requires iptables or tc (traffic control)
import { exec } from 'child_process'
import { promisify } from 'util'
const run = promisify(exec)

async function partitionAgents(ip1: string, ip2: string) {
  // Drop traffic in both directions
  await run(`iptables -A INPUT  -s ${ip2} -j DROP`)
  await run(`iptables -A OUTPUT -d ${ip2} -j DROP`)
  console.log(`[jepsen] partition: ${ip1} ↔ ${ip2} blocked`)
}

async function healPartition(ip1: string, ip2: string) {
  await run(`iptables -D INPUT  -s ${ip2} -j DROP`)
  await run(`iptables -D OUTPUT -d ${ip2} -j DROP`)
  console.log('[jepsen] partition healed')
}

Linearizability checker (knossos-style)

// Record all operations with their start/end times
interface Op {
  type:    'invoke' | 'ok' | 'fail'
  fn:      string
  value:   unknown
  process: string
  time:    number   // monotonic ms
}

const history: Op[] = []

async function trackedLock(agentId: string, lockFn: () => Promise<boolean>) {
  history.push({ type: 'invoke', fn: 'lock', value: null, process: agentId, time: Date.now() })
  try {
    const result = await lockFn()
    history.push({ type: 'ok',     fn: 'lock', value: result, process: agentId, time: Date.now() })
    return result
  } catch (err) {
    history.push({ type: 'fail',   fn: 'lock', value: null,   process: agentId, time: Date.now() })
    throw err
  }
}

// Check: no two agents hold lock simultaneously
function checkLinearizability(history: Op[]): boolean {
  const held = new Map<string, boolean>()
  const locks: string[] = []

  for (const op of history.filter(o => o.type === 'ok' && o.fn === 'lock' && o.value === true)) {
    locks.push(op.process)
    if (locks.length > 1) {
      console.error('[jepsen] VIOLATION: concurrent lock holders:', locks)
      return false
    }
  }
  return true
}

Clock skew injection

# Shift system clock forward 200ms to simulate NTP jump (requires root)
sudo date -s "$(date --date='+200 ms' '+%Y-%m-%d %H:%M:%S.%N')"

# tc: add 100ms jitter to all outgoing packets
sudo tc qdisc add dev eth0 root netem delay 100ms 50ms distribution normal

# Restore
sudo tc qdisc del dev eth0 root

Anti-Fake-Pass Checklist

❌ Testing on localhost only → no real network partition possible; tests prove nothing about distributed behavior
❌ Not recording op timestamps → linearizability check cannot detect overlapping lock windows
❌ Healing partition before checking consistency → find violations while partition is active
❌ Single-node "cluster" in tests → consensus algorithms behave differently with 1 vs 3 nodes
❌ iptables rules not cleaned up after test → permanent partition in CI environment
❌ Assuming test pass = production safe → Jepsen finds bugs in mature DBs (etcd, Cassandra, etc.)

04 同仓库更多 Skills

Horonomos-laps
Horonomos-laps/Yana-AI

openai--sentry--sentry

Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

ito-basket-compare

Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

implementing-network-traffic-baselining

Build network traffic baselines from NetFlow/IPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly/daily traffic pattern profiling

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

detecting-evasion-techniques-in-endpoint-logs

Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

implementing-microsegmentation-with-guardicore

Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.

★ 13
Horonomos-laps
Horonomos-laps/Yana-AI

ramdisk-tmpfs-patterns

暂无简介

★ 13