[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7CF5GD1TYFMW5AA020994XK":3,"skill-same:01M7CF5GD1TYFMW5AA020994XK":63},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":15,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":5,"forksCount":26,"repositoryHtmlUrl":27,"repositoryHomepageUrl":28,"ownerHtmlUrl":29,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":30,"install":31,"resources":44},"01M7CF5GD1TYFMW5AA020994XK","harness-first",null,"Diagnose and fix an unreliable, expensive, or unsafe LLM agent by auditing its harness (golden set, judge, cost caps, data layer, action approvals, tracing) before blaming or swapping the model. Use when someone says an agent is \"burning tokens\", \"hallucinating\", \"brittle\", gives inconsistent answers, asks whether to switch to a cheaper\u002Fbetter model, or wants to ship an agent or prompt change to customers.","getedgehq\u002Fskills","getedgehq","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F327350993?v=4","01M7CEH5YJQYF1PB3MDG9PA5HX","01M7CEH5ZAPY8TX4H3E04KM875",5,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fharness-first\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fharness-first\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: harness-first\ndescription: Diagnose and fix an unreliable, expensive, or unsafe LLM agent by auditing its harness (golden set, judge, cost caps, data layer, action approvals, tracing) before blaming or swapping the model. Use when someone says an agent is \"burning tokens\", \"hallucinating\", \"brittle\", gives inconsistent answers, asks whether to switch to a cheaper\u002Fbetter model, or wants to ship an agent or prompt change to customers.\n---\n\n# Harness first\n\nA model is an engine. Most \"model problems\" are a missing harness: nothing defines a correct output, nothing checks changes, nothing caps spend, the data has no agreed definitions, actions have no approval, and there are no traces to find the bad call. Audit that before touching the model.\n\n## 1. Reproduce the symptom from evidence, not vibes\n\n- Read the agent code the symptom touches: the loop, tool definitions, prompt assembly, data access, side-effecting tools.\n- Read whatever logs\u002Ftraces\u002Ftranscripts exist. Aggregate them (per conversation, per tool, per error): who spends the tokens, which calls fail, which answers are wrong. A few outliers usually dominate.\n- State the root cause as a mechanism with evidence (file:line, log ids, numbers you computed). \"The model is bad\" is not a mechanism.\n\nCommon mechanisms:\n- **Token burn:** loop with no max-iterations, retry on a deterministic error, full history or a huge tool result resent every turn, no caching of stable context.\n- **\"Hallucination\":** the agent faithfully used an ambiguous or wrong source (two tables\u002Ffields with the same name and different meaning, stale doc). Recompute both numbers yourself to prove it.\n- **Brittleness:** prompt changes shipped with no regression set; behavior that only the author checked by hand.\n\n## 2. Score the six harness parts\n\nFor each, mark **present \u002F partial \u002F missing** with the evidence you found:\n\n| Part | Present means |\n|---|---|\n| Golden set | ~20+ cases with expected answers\u002Fconstraints, drawn from real traffic, incidents, and edge cases |\n| Judge | deterministic checks first (numbers, policy rules, forbidden content), LLM-rubric only for what code can't check; runs on every prompt\u002Fmodel\u002Ftool change |\n| Cost governance | hard token\u002Fcost caps per user and per workflow, max iterations per run, graceful stop with a message |\n| Data layer | read-only credentials for reads, a data dictionary that defines each metric\u002Ffield the agent may use |\n| Action safety | irreversible or external actions (send, pay, delete, write, refund) are reversible or gated by human approval |\n| Tracing | every model and tool call logged with conversation id, workflow, input, output, tokens, cost, latency, error |\n\n## 3. Decide before building\n\n- **Never recommend a model swap without eval evidence.** If someone asks \"should we switch models?\", the answer is: fix the mechanism, then run the golden set on both models and compare quality and cost. Say what the fix alone is expected to save, with your numbers.\n- **Never approve shipping a prompt\u002Fagent change without a golden-set run.** If recorded outputs exist, score them against the expected answers yourself and report failures by case id.\n- Safety gaps (ungated side effects, write access on a read path, leaking internal data) are blocking, even when nobody asked about them.\n\n## 4. Build the minimum harness, in this order\n\n1. **Stop the bleeding:** max iterations, no retry on non-transient errors, truncate\u002Fsummarize large tool output, hard per-run cost cap.\n2. **Gate side effects:** approval step or draft mode for irreversible tools; read-only connection for query tools.\n3. **Golden set file** (`evals\u002Fgolden.jsonl` or similar): input, expected answer or constraints, source (log id \u002F incident \u002F policy clause). Include the cases that just failed.\n4. **Judge script** that runs the golden set and prints pass\u002Ffail per case plus totals. Deterministic checks before LLM grading.\n5. **Data dictionary** for every metric the agent reports (definition, source table\u002Ffield, exclusions).\n6. **Tracing** fields listed above, if missing.\n\nWrite real files, not a description of files. Run what can run and report the result.\n\n## 5. Report\n\n- One-line answer to the question actually asked (e.g. \"Don't switch models yet: 3 runs stuck in a retry loop spent 71% of tokens\").\n- Root cause with evidence and numbers.\n- Harness scorecard (the six parts).\n- What you changed or created, and what you ran.\n- Blocking risks, then a short prioritized next-steps list.\n\n## Credit\n\nThe method this Skill follows comes from a public post by Mark Ajzenstadt (@mardehaym): https:\u002F\u002Fx.com\u002Fmardehaym\u002Fstatus\u002F2099562230647804152. The argument is theirs; the procedure, the wording and the evaluation here are Edge's own, and the author did not review them.\n","# Harness first\n\nA model is an engine. Most \"model problems\" are a missing harness: nothing defines a correct output, nothing checks changes, nothing caps spend, the data has no agreed definitions, actions have no approval, and there are no traces to find the bad call. Audit that before touching the model.\n\n## 1. Reproduce the symptom from evidence, not vibes\n\n- Read the agent code the symptom touches: the loop, tool definitions, prompt assembly, data access, side-effecting tools.\n- Read whatever logs\u002Ftraces\u002Ftranscripts exist. Aggregate them (per conversation, per tool, per error): who spends the tokens, which calls fail, which answers are wrong. A few outliers usually dominate.\n- State the root cause as a mechanism with evidence (file:line, log ids, numbers you computed). \"The model is bad\" is not a mechanism.\n\nCommon mechanisms:\n- **Token burn:** loop with no max-iterations, retry on a deterministic error, full history or a huge tool result resent every turn, no caching of stable context.\n- **\"Hallucination\":** the agent faithfully used an ambiguous or wrong source (two tables\u002Ffields with the same name and different meaning, stale doc). Recompute both numbers yourself to prove it.\n- **Brittleness:** prompt changes shipped with no regression set; behavior that only the author checked by hand.\n\n## 2. Score the six harness parts\n\nFor each, mark **present \u002F partial \u002F missing** with the evidence you found:\n\n| Part | Present means |\n|---|---|\n| Golden set | ~20+ cases with expected answers\u002Fconstraints, drawn from real traffic, incidents, and edge cases |\n| Judge | deterministic checks first (numbers, policy rules, forbidden content), LLM-rubric only for what code can't check; runs on every prompt\u002Fmodel\u002Ftool change |\n| Cost governance | hard token\u002Fcost caps per user and per workflow, max iterations per run, graceful stop with a message |\n| Data layer | read-only credentials for reads, a data dictionary that defines each metric\u002Ffield the agent may use |\n| Action safety | irreversible or external actions (send, pay, delete, write, refund) are reversible or gated by human approval |\n| Tracing | every model and tool call logged with conversation id, workflow, input, output, tokens, cost, latency, error |\n\n## 3. Decide before building\n\n- **Never recommend a model swap without eval evidence.** If someone asks \"should we switch models?\", the answer is: fix the mechanism, then run the golden set on both models and compare quality and cost. Say what the fix alone is expected to save, with your numbers.\n- **Never approve shipping a prompt\u002Fagent change without a golden-set run.** If recorded outputs exist, score them against the expected answers yourself and report failures by case id.\n- Safety gaps (ungated side effects, write access on a read path, leaking internal data) are blocking, even when nobody asked about them.\n\n## 4. Build the minimum harness, in this order\n\n1. **Stop the bleeding:** max iterations, no retry on non-transient errors, truncate\u002Fsummarize large tool output, hard per-run cost cap.\n2. **Gate side effects:** approval step or draft mode for irreversible tools; read-only connection for query tools.\n3. **Golden set file** (`evals\u002Fgolden.jsonl` or similar): input, expected answer or constraints, source (log id \u002F incident \u002F policy clause). Include the cases that just failed.\n4. **Judge script** that runs the golden set and prints pass\u002Ffail per case plus totals. Deterministic checks before LLM grading.\n5. **Data dictionary** for every metric the agent reports (definition, source table\u002Ffield, exclusions).\n6. **Tracing** fields listed above, if missing.\n\nWrite real files, not a description of files. Run what can run and report the result.\n\n## 5. Report\n\n- One-line answer to the question actually asked (e.g. \"Don't switch models yet: 3 runs stuck in a retry loop spent 71% of tokens\").\n- Root cause with evidence and numbers.\n- Harness scorecard (the six parts).\n- What you changed or created, and what you ran.\n- Blocking risks, then a short prioritized next-steps list.\n\n## Credit\n\nThe method this Skill follows comes from a public post by Mark Ajzenstadt (@mardehaym): https:\u002F\u002Fx.com\u002Fmardehaym\u002Fstatus\u002F2099562230647804152. The argument is theirs; the procedure, the wording and the evaluation here are Edge's own, and the author did not review them.\n",{"name":5,"description":7},"main","01960aeb08909dbd0c1a2d4f219cdb300099b8b9",0,"https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills","https:\u002F\u002Fgetedge.cc","https:\u002F\u002Fgithub.com\u002Fgetedgehq","2026-11-12 10:05:02",{"source":8,"skillName":5,"commands":32,"prompt":42,"zipAvailable":43},[33,36,39],{"tool":34,"command":35},"npx","npx skills add getedgehq\u002Fskills --skill harness-first",{"tool":37,"command":38},"bunx","bunx skills add getedgehq\u002Fskills --skill harness-first",{"tool":40,"command":41},"pnpm","pnpm dlx skills add getedgehq\u002Fskills --skill harness-first","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fharness-first\u002FSKILL.md",true,[45,51,55,59],{"path":46,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":49,"sourceHtmlUrl":50},"agents\u002Fopenai.yaml","REFERENCE","参考文档",285,"https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fharness-first\u002Fagents\u002Fopenai.yaml",{"path":52,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":53,"sourceHtmlUrl":54},"DERIVATION.json",2749,"https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fharness-first\u002FDERIVATION.json",{"path":56,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":57,"sourceHtmlUrl":58},"EVALS.md",27123,"https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fharness-first\u002FEVALS.md",{"path":60,"resourceType":60,"resourceTypeDesc":61,"sizeBytes":62,"sourceHtmlUrl":15},"LICENSE","许可证",10756,[64,72,80,87,94,102],{"skillNo":65,"name":66,"nameZh":6,"description":67,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":68,"sourceHtmlUrl":69,"categories":70,"keywords":71,"publishedAt":19},"01M7CFVPC1P7JN60J5AN9AZZ5G","zero-downtime-cloud-move","Runbook for moving a live app from one cloud to another (Google Cloud to AWS, AWS to Google Cloud, a PaaS to your own account, or one storage provider to another) without downtime. The agent inventories what is really running, builds the new stack next to the old one, keeps a one-line rollback for every phase, proves the new stack with real workloads before any traffic moves, cuts over with a checklist, watches a fixed window after the switch, copies data with an idempotent job whose second pass must report 0 left, and deletes the old cloud last. Use when someone asks to migrate, move or replatform a production app, switch cloud providers, move a backend, database files or object storage, plan a DNS cutover, or decommission an old cloud account.","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fzero-downtime-cloud-move\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fzero-downtime-cloud-move\u002FSKILL.md",[],[],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":76,"sourceHtmlUrl":77,"categories":78,"keywords":79,"publishedAt":19},"01M7CFVH8VN6G2KE8PBFD1CRMN","zayats-ceo-branding","Build an evidence-based CEO positioning paper, Personal Branding Roadmap, and editorial workflow using Marina Zayats's public method. Use for executive branding, LinkedIn ghostwriting briefs, profile strategy, and recurring CEO-approved content planning.","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fzayats-ceo-branding\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fzayats-ceo-branding\u002FSKILL.md",[],[],{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":19},"01M7CFV9BB7G8K8CX9RKZ3GECX","yc-app-roast","YC App Roast: roast a YC or a16z Speedrun application. Use when a founder wants their YC or Speedrun answers scored, asks \"what would a YC partner think\", wants the weak answers found or rewritten, or wants their draft checked before they submit. Scores every answer 0-10 against public advice from YC and Speedrun partners, names the top issue with the exact words that cost points, and writes a rewrite from the founder's own facts. Works for YC and a16z Speedrun. Not affiliated with Y Combinator or a16z.","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fyc-app-roast\u002FSKILL.md",[],[],{"skillNo":88,"name":89,"nameZh":6,"description":90,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":91,"categories":92,"keywords":93,"publishedAt":19},"01M7CFV11Z1HB9EEWFQ6PCJ98X","yc-app-review","YC App Review: get a YC or a16z Speedrun application reviewed by people you trust. Use when a founder wants friends, mentors or alumni to review their YC or Speedrun application, or asks to share their draft for feedback. Builds a shareable review page (one HTML file, application-portal look) where reviewers leave notes per question and press \"Send my notes\" to send them back as text, then parses the notes into proposed rewrites. Optional tracked private links per reviewer. Works for YC and a16z Speedrun. Not affiliated with Y Combinator or a16z.","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fyc-app-review\u002FSKILL.md",[],[],{"skillNo":95,"name":96,"nameZh":6,"description":97,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":98,"sourceHtmlUrl":99,"categories":100,"keywords":101,"publishedAt":19},"01M7CFTSVJV7XCSW62W39MX8CZ","x-growth-playbook","X Growth Playbook. Analyse a person's own X (Twitter) posts and tell them what to post next, with the numbers behind every claim. Compares quote posts vs originals, links vs no links, short vs long, time of day in their timezone, media vs text and the topics they name, reports medians with n, then writes a 10-step plan and the next 7 posts in their voice. Use when the user says \"analyse my X posts\", \"what works on my Twitter\", \"how do I grow on X\", \"what should I post next on X\", \"audit my X account\", or shares an X data archive, X API export or a list of their posts. Runs on the user's computer.","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fx-growth-playbook\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fx-growth-playbook\u002FSKILL.md",[],[],{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":106,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":19},"01M7CFTK101S3P00MQ129G5YNH","workplan","Create, update, or close work plans for multi-step tasks. Use when starting refactors, bug lists, feature work, migrations, or any task with 2+ steps. Also use after auto-compaction to re-orient. Triggers: \"workplan\", \"work plan\", \"create a plan\", \"what's the plan\", \"where was I\", or when Claude detects a multi-step task that needs tracking.","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fworkplan\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fgetedgehq\u002Fskills\u002Fblob\u002Fmain\u002Fworkplan\u002FSKILL.md",[],[]]