AGENT SKILL

smoke-test

Quick stress-test smoke validation for stablehand — verifies each harness and model works

License 许可
MIT
所属仓库
iofold/stablehand
最近更新
2026-10-25
Stars
★ 15
Forks
2

01 前言 Front Matter

name
smoke-test
description
Quick stress-test smoke validation for stablehand — verifies each harness and model works

03 SKILL.md 正文

Stress smoke test

Run from the repo root with Docker/API already up.

1) Health check

set -euo pipefail
API_BASE="http://localhost:3002"
AUTH_HEADER=()
if [[ -n "${STABLEHAND_API_TOKEN:-}" ]]; then
  AUTH_HEADER=(-H "Authorization: Bearer $STABLEHAND_API_TOKEN")
fi

curl -s "$API_BASE/health" | jq .

Pass if the response is HTTP 200 and shows a healthy service payload.

2) Seed stress agents

npx tsx tests/stress/seed.ts

Capture agent ids for later calls:

AGENTS_JSON="$(curl -s "${AUTH_HEADER[@]}" "$API_BASE/api/agents")"

agent_id() {
  local name="$1"
  jq -r --arg name "$name" 'first(.[] | select(.name == $name) | .id)' <<<"$AGENTS_JSON"
}

LG_OPENAI_ID="$(agent_id stress-lg-openai)"
LG_GEMINI_ID="$(agent_id stress-lg-gemini)"
PI_OPENAI_ID="$(agent_id stress-pi-openai)"
PI_GEMINI_ID="$(agent_id stress-pi-gemini)"
CLAUDE_ID="$(agent_id stress-claude-sonnet)"
CHAIN_ID="$(agent_id stress-chain-responder)"
HITL_ID="$(agent_id stress-hitl)"

printf '%s\n' \
  "stress-lg-openai=$LG_OPENAI_ID" \
  "stress-lg-gemini=$LG_GEMINI_ID" \
  "stress-pi-openai=$PI_OPENAI_ID" \
  "stress-pi-gemini=$PI_GEMINI_ID" \
  "stress-claude-sonnet=$CLAUDE_ID" \
  "stress-chain-responder=$CHAIN_ID" \
  "stress-hitl=$HITL_ID"

Pass if all 7 ids are non-empty.

3) Send one message to each agent and wait for completion

Reusable polling helper:

post_message() {
  local agent_id="$1"
  local content="$2"
  curl -s -X POST "$API_BASE/api/agents/$agent_id/message" \
    "${AUTH_HEADER[@]}" \
    -H 'Content-Type: application/json' \
    -d "$(jq -nc --arg content "$content" '{content: $content}')" \
    | jq -er '.turnId'
}

wait_for_status() {
  local agent_id="$1"
  local turn_id="$2"
  local wanted="$3"

  while true; do
    status="$(curl -s "${AUTH_HEADER[@]}" "$API_BASE/api/agents/$agent_id/turns" \
      | jq -r --arg turn_id "$turn_id" 'first(.turns[]? | select(.id == $turn_id) | .status) // empty' \
      | tr '[:upper:]' '[:lower:]')"
    echo "turn=$turn_id status=${status:-missing}"

    case "$status" in
      "$wanted") return 0 ;;
      error|failed|cancelled|rejected) return 1 ;;
    esac

    sleep 2
  done
}

wait_for_interrupt() {
  local agent_id="$1"
  local turn_id="$2"

  while true; do
    interrupt_id="$(curl -s "${AUTH_HEADER[@]}" "$API_BASE/api/agents/$agent_id/interrupts" \
      | jq -r --arg turn_id "$turn_id" 'first(.interrupts[]? | select(.turnId == $turn_id) | .interruptId) // empty')"
    [[ -n "$interrupt_id" ]] && return 0

    status="$(curl -s "${AUTH_HEADER[@]}" "$API_BASE/api/agents/$agent_id/turns" \
      | jq -r --arg turn_id "$turn_id" 'first(.turns[]? | select(.id == $turn_id) | .status) // empty' \
      | tr '[:upper:]' '[:lower:]')"
    echo "turn=$turn_id status=${status:-missing} waiting_for_interrupt"
    [[ "$status" =~ ^(completed|error|failed|cancelled|rejected)$ ]] && return 1
    sleep 2
  done
}

stress-lg-openai

LG_OPENAI_TURN="$(post_message "$LG_OPENAI_ID" "Quick smoke test: use a basic tool, finish cleanly, and emit stress.tools.complete.")"

wait_for_status "$LG_OPENAI_ID" "$LG_OPENAI_TURN" completed

stress-lg-gemini

LG_GEMINI_TURN="$(post_message "$LG_GEMINI_ID" "Quick smoke test: verify Gemini tool execution and emit stress.gemini.complete.")"

wait_for_status "$LG_GEMINI_ID" "$LG_GEMINI_TURN" completed

stress-pi-openai

PI_OPENAI_TURN="$(post_message "$PI_OPENAI_ID" "Quick smoke test: run once and emit stress.pi.complete.")"

wait_for_status "$PI_OPENAI_ID" "$PI_OPENAI_TURN" completed

stress-pi-gemini

PI_GEMINI_TURN="$(post_message "$PI_GEMINI_ID" "Quick smoke test: run once and emit stress.pi-gemini.complete.")"

wait_for_status "$PI_GEMINI_ID" "$PI_GEMINI_TURN" completed

stress-claude-sonnet

CLAUDE_TURN="$(post_message "$CLAUDE_ID" "Quick smoke test: write and verify a tiny file, then emit stress.claude.complete.")"

wait_for_status "$CLAUDE_ID" "$CLAUDE_TURN" completed

stress-chain-responder

CHAIN_TURN="$(post_message "$CHAIN_ID" "Quick smoke test: acknowledge the request and emit stress.chain.done.")"

wait_for_status "$CHAIN_ID" "$CHAIN_TURN" completed

stress-hitl

HITL_TURN="$(post_message "$HITL_ID" "Create the approval smoke file now.")"

wait_for_interrupt "$HITL_ID" "$HITL_TURN"

4) Approve HITL interrupt

INTERRUPT_ID="$(curl -s "${AUTH_HEADER[@]}" "$API_BASE/api/agents/$HITL_ID/interrupts" \
  | jq -er --arg turn_id "$HITL_TURN" 'first(.interrupts[]? | select(.turnId == $turn_id) | .interruptId)')"

curl -s -X POST "$API_BASE/api/agents/$HITL_ID/interrupt" \
  "${AUTH_HEADER[@]}" \
  -H 'Content-Type: application/json' \
  -d "$(jq -nc --arg action approve --arg interruptId "$INTERRUPT_ID" '{action: $action, interruptId: $interruptId}')" \
  | jq .

wait_for_status "$HITL_ID" "$HITL_TURN" completed

Pass if the interrupt resolves successfully and the HITL turn ends in completed.

5) Check completion events

curl -s "${AUTH_HEADER[@]}" "$API_BASE/api/events?type=stress.tools.complete&limit=20" | jq '.events'

Pass if at least one stress.tools.complete event exists from this run.

Post-run cleanup (manual, after human verification)

Do NOT run cleanup as part of the test. Cleanup cascade-deletes events, turns, conversations, and messages.

Run only after the human has reviewed all results:

npx tsx tests/stress/seed.ts --clean

04 同仓库更多 Skills