AGENT SKILL

benchmark-optimization-loop

Use when the user asks to make something faster, try many variants, run recursive optimization, benchmark latency/throughput/cost, or choose the best implementation by repeated measured tests.

License 许可
MIT
最近更新
2026-11-15
Stars
★ 0
Forks
0

01 前言 Front Matter

name
benchmark-optimization-loop
tools
Read, Write, Edit, Bash, Grep, Glob
license
MIT
metadata
{ "origin": "ECC" }
description
Use when the user asks to make something faster, try many variants, run recursive optimization, benchmark latency/throughput/cost, or choose the best implementation by repeated measured tests.

03 SKILL.md 正文

Benchmark Optimization Loop

Use this skill to convert "make it 20x faster" or "try 50 recursive optimizations" into a bounded measured loop that can actually improve a system.

Required Baseline

Do not optimize until these exist:

  • the operation being optimized;
  • the correctness gate that must stay green;
  • the metric: wall time, p95 latency, rows/sec, cost/run, memory, error rate;
  • the current baseline;
  • the search budget: max variants, max time, max spend, max data impact.

If the user asks for an unrealistic target, keep the ambition but make the loop bounded and measurable.

Loop

  1. Measure the baseline.
  2. Identify bottlenecks from evidence.
  3. Generate variants that test one hypothesis each.
  4. Run variants with the same input shape.
  5. Reject variants that fail correctness, safety, or reproducibility.
  6. Promote the fastest safe variant.
  7. Codify the winning path in a script, command, test, config, or doc.
  8. Rerun the baseline and winner to confirm the delta.

Variant Table

Track variants like this:

Variant | Hypothesis | Command | Time | Correct? | Notes
baseline | current path | npm run job | 120s | yes | stable
batch-500 | fewer round trips | npm run job -- --batch 500 | 42s | yes | winner
parallel-8 | more workers | npm run job -- --workers 8 | 31s | no | rate limited

Recursive Search

For recursive or hyperparameter work:

  • persist every run to a ledger;
  • compare against the prior accepted winner, not only the previous run;
  • keep a holdout or replay check;
  • stop when improvement is within noise, correctness fails, cost exceeds the budget, or the search starts changing more variables than it can explain.

Use phrases like "best measured safe variant" instead of "global optimum" unless the search space was actually exhaustive.

Promotion Gate

A variant cannot become the new default until:

  • correctness tests pass;
  • the performance delta is repeated or explained;
  • rollback is obvious;
  • the change is encoded in source control or a durable runbook;
  • the final summary includes exact commands and measurements.

04 同仓库更多 Skills

Henrique1601
Henrique1601/SkiilsToIAs

rag-architect-agent

Agente Arquiteto de RAG, Embeddings & Busca Vetorial. Especialista em design e implementação de pipelines de Retrieval-Augmented Generation de alta fidelidade: chunking semântico, modelagem pgvector no PostgreSQL (Neon/Supabase), busca híbrida (cosseno + BM25), reranking, validação via TypeSafe AI e filosofia Ponytail (anti-overengineering).

★ 0
Henrique1601
Henrique1601/SkiilsToIAs

rag-architecture-patterns

Padrões arquiteturais e implementação de pipelines RAG (Retrieval-Augmented Generation) de alta fidelidade para qualquer projeto ou sistema. Cobre chunking semântico, embeddings, busca híbrida (pgvector + BM25), reranking, validação via TypeSafe AI e filosofia Ponytail (anti-overengineering).

★ 0
Henrique1601
Henrique1601/SkiilsToIAs

skill-repair

Use this to fix and re-install agent skills that have failed installation. This skill provides the necessary context and permissions to surgically update the `manifest.json` after a fix has been applied.

★ 0
Henrique1601
Henrique1601/SkiilsToIAs

security-scan

Scan your Claude Code configuration (.claude/ directory) for security vulnerabilities, misconfigurations, and injection risks using AgentShield. Checks CLAUDE.md, settings.json, MCP servers, hooks, and agent definitions.

★ 0
Henrique1601
Henrique1601/SkiilsToIAs

security-review

Use this skill when adding authentication, handling user input, working with secrets, creating API endpoints, or implementing payment/sensitive features. Provides comprehensive security checklist and patterns.

★ 0
Henrique1601
Henrique1601/SkiilsToIAs

security-bounty-hunter

Hunt for exploitable, bounty-worthy security issues in repositories. Focuses on remotely reachable vulnerabilities that qualify for real reports instead of noisy local-only findings.

★ 0