x-evaluate-runtime
Evaluate runtime protection controls: rate limiting, WAF rules, bot protection, DDoS mitigation, account lockout, brute force protection, CSP enforcement, and permissions policy. Produce SARIF 2.1.0 output with ASVS compliance mapping and scored Markdown report.
- License 许可
- MIT
- 最近更新
- 2026-10-27
- Stars
- ★ 0
- Forks
- 1
01 前言 Front Matter
- name
- x-evaluate-runtime
- description
- Evaluate runtime protection controls: rate limiting, WAF rules, bot protection, DDoS mitigation, account lockout, brute force protection, CSP enforcement, and permissions policy. Produce SARIF 2.1.0 output with ASVS compliance mapping and scored Markdown report.
- allowed-tools
- Read, Write, Bash, Glob, Grep
- argument-hint
- --target <url> [--scope all|rate-limit|waf|bot-protection|account-lockout|brute-force|csp|permissions] [--intensity passive|moderate|aggressive] [--login-endpoint /path]
- context-budget
- medium
- user-invocable
- true
- requires-capabilities
- []
02 资源文件 1
- 参考文档README.md2.2 KB
03 SKILL.md 正文
Global Output Policy
- Language: English ONLY.
- Tone: Technical, Direct, and Concise.
- Efficiency: Remove all conversational fillers and greetings to save tokens.
Skill: Runtime Protection Eval
Purpose
Evaluate runtime protection controls for {{PROJECT_NAME}} by analyzing active defense mechanisms including rate limiting, WAF rules, bot protection, account lockout, brute force mitigation, CSP enforcement, and permissions policy. Produce SARIF 2.1.0 output with ASVS compliance mapping and a scored Markdown report.
Triggers
/x-evaluate-runtime --target https://app.example.com— evaluate all dimensions/x-evaluate-runtime --target https://app.example.com --scope rate-limit— rate limiting only/x-evaluate-runtime --target https://app.example.com --scope waf— WAF rules only/x-evaluate-runtime --target https://app.example.com --scope account-lockout --login-endpoint /api/auth/login— account lockout/x-evaluate-runtime --target https://app.example.com --intensity passive— observe headers only/x-evaluate-runtime --target https://app.example.com --intensity aggressive— full volume testing (local/dev only)
Parameters
| Parameter | Type | Required | Default | Validation | Example |
|---|---|---|---|---|---|
--target |
URL | Yes | — | Valid HTTP/HTTPS URL | https://app.example.com |
--scope |
Enum | No | all |
all, rate-limit, waf, bot-protection, account-lockout, brute-force, csp, permissions |
rate-limit |
--intensity |
Enum | No | moderate |
passive, moderate, aggressive |
passive |
--login-endpoint |
Path | No | — | Relative path starting with / |
/api/auth/login |
Workflow
Step 1 — Validate Parameters
Validate all inputs before execution:
--targetmust be a valid HTTP or HTTPS URL--targetmust be reachable (TCP connection within 10s timeout)- If target is unreachable, emit error
"Target unreachable: {url}"and abort --scopemust match one of the 8 valid values--login-endpointis required when--scope=account-lockout
Step 2 — Configure Intensity
Resolve effective intensity:
- If
--intensity=aggressiveand environment is production:- Log warning:
"Aggressive intensity not allowed in production" - Downgrade to
passive
- Log warning:
- Record effective intensity in report metadata
Intensity levels:
| Level | Behavior | Allowed Environments |
|---|---|---|
| passive | Observe headers and configurations only; no payloads sent | All environments |
| moderate | Send non-destructive test payloads (default) | All environments |
| aggressive | Test limits with higher volume of requests | Local and dev only |
Step 3 — Discover Baseline
Probe the target to establish baseline:
- Send a single GET request to
--target - Record response headers for later comparison
- Extract security-relevant headers:
X-RateLimit-*,Content-Security-Policy,Permissions-Policy,X-Content-Type-Options,Strict-Transport-Security - Record server identification if exposed
Step 4 — Evaluate Dimensions
Execute checks for each dimension in scope. Each dimension produces a RuntimeProtectionResult.
4.1 — Rate Limiting
ASVS Reference: V4.3 (Access Control)
| Check | Passive | Moderate | Aggressive |
|---|---|---|---|
X-RateLimit-* headers present |
Yes | Yes | Yes |
Retry-After header on 429 |
No | Yes | Yes |
| Global rate limit triggers 429 | No | Yes | Yes |
| Per-endpoint rate limit | No | Yes | Yes |
| Per-user rate limit | No | No | Yes |
| Burst handling | No | No | Yes |
Moderate test procedure:
- Send burst of 100 requests to target root endpoint within 10 seconds
- Count responses with status 429
- Validate
Retry-Afterheader is present on 429 responses - Check for
X-RateLimit-RemainingandX-RateLimit-Limitheaders
Result fields:
| Field | Type | Description |
|---|---|---|
| endpoint | String | Tested endpoint path |
| requestsSent | int | Total requests sent |
| requestsBlocked | int | Responses with 429 status |
| statusCode | int | Expected blocking status (429) |
| retryAfterPresent | boolean | Whether Retry-After header was found |
| rateLimitHeaders | Map | Rate limit headers observed |
4.2 — WAF Rules (OWASP CRS)
ASVS Reference: V5.1 (Input Validation)
| Check | Passive | Moderate | Aggressive |
|---|---|---|---|
| Security headers indicating WAF | Yes | Yes | Yes |
| SQL injection blocking | No | Yes | Yes |
| XSS payload blocking | No | Yes | Yes |
| Path traversal blocking | No | Yes | Yes |
| Custom WAF rules | No | No | Yes |
Non-destructive test payloads (moderate):
SQL Injection: GET /search?q=' OR '1'='1
XSS: GET /search?q=<script>alert(1)</script>
Path Traversal: GET /files/../../../etc/passwd
Command Inject: GET /ping?host=;cat /etc/passwd
These payloads probe for WAF blocking behavior (expecting 403 or custom block page) without causing harm.
Status: PROTECTED if all payloads return 403/blocking response; PARTIAL if some blocked; UNPROTECTED if none blocked; SKIPPED in passive mode.
4.3 — Bot Protection
ASVS Reference: V13.1 (API and Web Service)
| Check | Passive | Moderate | Aggressive |
|---|---|---|---|
| CAPTCHA presence in HTML | Yes | Yes | Yes |
| Bot detection headers | Yes | Yes | Yes |
| Automated request blocking | No | Yes | Yes |
| Challenge page detection | No | Yes | Yes |
Checks:
- Scan HTML responses for CAPTCHA markers (reCAPTCHA, hCaptcha, Turnstile)
- Check for bot-detection response headers
- Send requests with missing/invalid User-Agent
- Detect challenge/interstitial pages
4.4 — Account Lockout
ASVS Reference: V2.2 (Authentication)
| Check | Passive | Moderate | Aggressive |
|---|---|---|---|
| Login endpoint responds | Yes | Yes | Yes |
| Lockout after max attempts | No | Yes | Yes |
| Lockout duration enforced | No | Yes | Yes |
| Progressive delay detection | No | No | Yes |
| Lockout notification | No | No | Yes |
| Reset mechanism available | No | No | Yes |
Requires: --login-endpoint parameter.
Moderate test procedure:
- Send 10 login attempts with invalid credentials to
--login-endpoint - Check for 423 (Locked) or 429 (Too Many Requests) response
- Validate lockout message in response body
- Record number of attempts before lockout
Finding on absence: If no lockout is detected after 10 failed attempts, generate a CRITICAL finding with fix recommendation including max attempts and lockout duration configuration.
4.5 — Brute Force Protection
ASVS Reference: V2.2 (Authentication), V11.1 (Business Logic)
| Check | Passive | Moderate | Aggressive |
|---|---|---|---|
| Rate limiting on auth endpoints | Yes | Yes | Yes |
| Progressive delay observed | No | Yes | Yes |
| API key enumeration protection | No | Yes | Yes |
| Timing attack mitigation | No | Yes | Yes |
Checks:
- Verify rate limiting on authentication endpoints (separate from global rate limit)
- Measure response time variance for valid vs. invalid credentials (timing attack mitigation expects constant time)
- Send sequential requests with incrementing API key patterns to detect enumeration protection
4.6 — CSP Enforcement
ASVS Reference: V14.4 (Configuration)
| Check | Passive | Moderate | Aggressive |
|---|---|---|---|
Content-Security-Policy header present |
Yes | Yes | Yes |
| CSP directives completeness | Yes | Yes | Yes |
report-uri or report-to configured |
Yes | Yes | Yes |
unsafe-inline absent |
Yes | Yes | Yes |
unsafe-eval absent |
Yes | Yes | Yes |
CSP directives to validate:
| Directive | Expected | Severity if Missing |
|---|---|---|
default-src |
Present and restrictive | HIGH |
script-src |
No unsafe-inline or unsafe-eval |
CRITICAL |
style-src |
Defined | MEDIUM |
img-src |
Defined | LOW |
connect-src |
Defined | MEDIUM |
frame-ancestors |
'none' or specific origins |
HIGH |
report-uri / report-to |
Present | MEDIUM |
4.7 — Permissions Policy
ASVS Reference: V14.4 (Configuration)
| Check | Passive | Moderate | Aggressive |
|---|---|---|---|
Permissions-Policy header present |
Yes | Yes | Yes |
| Feature restrictions defined | Yes | Yes | Yes |
| Geolocation restricted | Yes | Yes | Yes |
| Camera/microphone restricted | Yes | Yes | Yes |
| Payment API restricted | Yes | Yes | Yes |
Features to validate:
| Feature | Expected | Severity if Unrestricted |
|---|---|---|
geolocation |
() (disabled) or specific origins |
MEDIUM |
camera |
() (disabled) |
HIGH |
microphone |
() (disabled) |
HIGH |
payment |
() (disabled) or specific origins |
HIGH |
usb |
() (disabled) |
MEDIUM |
autoplay |
Restricted | LOW |
Step 5 — Score Results
5.1 — Per-Dimension Scoring
For each dimension, calculate:
dimensionScore = (passedChecks / totalChecks) * 100
Status assignment:
| Score Range | Status |
|---|---|
| 80-100 | PROTECTED |
| 40-79 | PARTIAL |
| 0-39 | UNPROTECTED |
| N/A | SKIPPED |
5.2 — Overall Scoring
overallScore = max(0, 100 - sum(severityWeight * findingCount))
Severity weights: CRITICAL=10, HIGH=5, MEDIUM=2, LOW=1, INFO=0
Grade assignment:
| Score | Grade |
|---|---|
| 90-100 | A |
| 80-89 | B |
| 70-79 | C |
| 60-69 | D |
| 0-59 | F |
Step 6 — Generate Reports
Generate two output files:
6.1 — SARIF 2.1.0 Output
Write to results/security/runtime-protection-{timestamp}.sarif.json:
{
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/main/sarif-2.1/schema/sarif-schema-2.1.0.json",
"version": "2.1.0",
"runs": [{
"tool": {
"driver": {
"name": "x-evaluate-runtime",
"version": "1.0.0",
"informationUri": "https://owasp.org/www-project-application-security-verification-standard/",
"rules": [
{
"id": "RTPROT-001",
"shortDescription": { "text": "Rate limit not configured" },
"helpUri": "https://cheatsheetseries.owasp.org/cheatsheets/Denial_of_Service_Cheat_Sheet.html",
"properties": { "asvs-ref": "V4.3", "dimension": "rate-limit" }
}
]
}
},
"results": [
{
"ruleId": "RTPROT-001",
"level": "error",
"message": { "text": "No rate limiting detected on endpoint /api/users" },
"locations": [{ "physicalLocation": { "artifactLocation": { "uri": "https://app.example.com/api/users" } } }],
"properties": {
"dimension": "rate-limit",
"asvs-ref": "V4.3",
"severity": "CRITICAL",
"fix-recommendation": "Configure rate limiting (e.g., 100 req/min per IP) with 429 response and Retry-After header"
}
}
]
}]
}
6.2 — Markdown Report
Write to results/security/runtime-protection-{timestamp}-report.md:
# Runtime Protection Report — {{PROJECT_NAME}}
**Date:** YYYY-MM-DD
**Target:** {target_url}
**Scope:** {scope}
**Intensity:** {effective_intensity}
## Overall Score: {score}/100 (Grade {grade})
## Dimension Summary
| Dimension | Status | Score | Checks | Passed | ASVS Ref |
|-----------|--------|-------|--------|--------|----------|
| Rate Limiting | PROTECTED | 90 | 6 | 5 | V4.3 |
| WAF Rules | PARTIAL | 60 | 5 | 3 | V5.1 |
| Bot Protection | UNPROTECTED | 20 | 4 | 1 | V13.1 |
| Account Lockout | SKIPPED | — | — | — | V2.2 |
| Brute Force | PROTECTED | 85 | 4 | 3 | V2.2, V11.1 |
| CSP Enforcement | PARTIAL | 70 | 5 | 3 | V14.4 |
| Permissions Policy | PROTECTED | 100 | 5 | 5 | V14.4 |
## Critical Gaps
- {dimension}: {description of gap and remediation}
## Detailed Findings
### [RTPROT-001] {Dimension} — {Finding Title}
- **Severity:** CRITICAL
- **ASVS Reference:** V4.3
- **Description:** {detailed description}
- **Evidence:** {observed behavior}
- **Fix Recommendation:** {actionable fix steps}
## Recommendations
1. **Immediate:** Address CRITICAL findings ({count})
2. **Short-term:** Resolve HIGH findings ({count})
3. **Long-term:** Review MEDIUM/LOW findings ({count})
ASVS Mapping Summary
| Dimension | ASVS Chapter | ASVS Section | Key Requirements |
|---|---|---|---|
| Rate Limiting | V4 Access Control | V4.3 | Rate limiting, resource protection |
| WAF Rules | V5 Validation | V5.1 | Input validation, injection prevention |
| Bot Protection | V13 API Security | V13.1 | API security mechanisms |
| Account Lockout | V2 Authentication | V2.2 | Credential protection, lockout |
| Brute Force | V2 Authentication, V11 Business Logic | V2.2, V11.1 | Brute force mitigation |
| CSP Enforcement | V14 Configuration | V14.4 | HTTP security headers |
| Permissions Policy | V14 Configuration | V14.4 | Feature policy headers |
Error Handling
| Scenario | Action |
|---|---|
| Target unreachable | Emit "Target unreachable: {url}", abort with no score |
| Connection timeout | Retry once with doubled timeout, then report error |
| Target returns 5xx | Report as finding (server error under load) |
| Login endpoint not provided for account-lockout scope | Warn and skip account-lockout dimension |
| Aggressive in production | Downgrade to passive, emit warning |
| Partial dimension failure | Complete other dimensions, mark failed as SKIPPED |
| SSL certificate error | Warn and continue with --insecure flag |
Integration Notes
| Skill | Relationship | Context |
|---|---|---|
| x-generate-security-dashboard | called-by | Dashboard aggregates runtime protection results |
| x-evaluate-hardening | complements | Hardening evaluates static config; runtime evaluates live behavior |
| x-run-dast | complements | DAST tests vulnerabilities; runtime tests defensive controls |
Knowledge Pack References
| Knowledge Pack | Usage |
|---|---|
| security | OWASP ASVS compliance mapping, remediation recommendations |
04 同仓库更多 Skills
x-parallel-eval
Detects and classifies file-collision risks between work units (tasks, stories, or epic phases) before parallel execution. Reads ## File Footprint blocks produced by x-task-plan / x-story-plan, applies the parallelism-heuristics knowledge pack (hard / regen / soft categories + hotspot overrides), and emits a collision matrix + serialization recommendation in Markdown (default) or JSON.
parallelism-heuristics
Canonical catalog of file-collision heuristics for parallel task, story, and epic execution. Defines the File Footprint block format, the three Conflict Categories (hard / regen / soft), a hotspot list of high-contention paths, and the degrade-with-warning policy. Consumed by x-task-plan, x-story-plan, x-epic-map, x-parallel-eval, and x-dev-*-implement.
x-watch-pr-ci
Polls a PR's CI checks and Copilot review status, blocking until checks complete or timeout. Returns one of 8 stable exit codes (SUCCESS=0, CI_PENDING_PROCEED=10, CI_FAILED=20, TIMEOUT=30, PR_ALREADY_MERGED=40, NO_CI_CONFIGURED=50, PR_CLOSED=60, PR_NOT_FOUND=70). Writes a versioned state-file for session resume.
x-validate-docs
Documentation freshness gate: validates 6 dimensions (readme, api-specs, grpc-proto, adr, skill-docs, system-architecture) against code changes in a PR. Stack-aware — only validates targets declared/auto-detected from ProjectConfig.documentation.targets. Returns exit non-zero on staleness; produces structured report.
x-update-system-architecture
Incrementally updates docs/architecture/system.md after an epic completes. Appends new entries to the Decision Log (§11) using x-internal-write-report --append (dedup by ## ID: marker) and surgically inserts component/integration changes into sections 1-10 via Edit. Idempotent: re-running with the same epic produces a byte-identical system.md.
x-update-architecture
Incrementally updates the service architecture document with changes from architecture plans. Adds new components, integrations, flows, and ADR references without rewriting existing content. Use after implementation to keep architecture documentation current.