[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7AW2ENC404T4V94FVQJMC68":3,"skill-same:01M7AW2ENC404T4V94FVQJMC68":69},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":14,"repositoryHtmlUrl":27,"repositoryHomepageUrl":6,"ownerHtmlUrl":28,"ownerBlogUrl":6,"ownerDescription":29,"updatedAt":30,"install":31,"resources":44},"01M7AW2ENC404T4V94FVQJMC68","self-evaluating-ai-pipeline",null,"Architect, implement, and automate 3-layer LLM evaluation pipelines (deterministic checks, anchored LLM-as-judge scoring, and human calibration) with golden regression suites and paired t-test statistical significance gating. Do not use for deterministic pure-function unit tests or ungrounded single-turn prompt drafting.","SpectreDeath\u002FBrain-Harness","SpectreDeath","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F207463416?v=4","01M7AVGR1D4X8ADT47APKGVH2Z","JL","01M7AVGR2HSGD23BA5DBTJJTVN",0,"MIT","https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fself-evaluating-ai-pipeline\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: self-evaluating-ai-pipeline\ndescription: Architect, implement, and automate 3-layer LLM evaluation pipelines (deterministic checks, anchored LLM-as-judge scoring, and human calibration) with golden regression suites and paired t-test statistical significance gating. Do not use for deterministic pure-function unit tests or ungrounded single-turn prompt drafting.\n---\n\n# Self-Evaluating AI Pipeline: Automated LLM Quality Architecture\n\nThe `self-evaluating-ai-pipeline` skill provides the definitive engineering framework for evaluating non-deterministic Large Language Model applications. Traditional software unit tests (`assert output == expected`) fail because LLM outputs vary across runs. This skill implements an opinionated, production-grade 3-layer evaluation funnel, regression suites with golden datasets, and statistical significance testing.\n\nSynthesized from Jude Otine's foundational literature (*How to Build a Self-Evaluating AI System*, freeCodeCamp, 2026), this pipeline enforces the **Shu-Ha-Ri framework principle**: moving from rigid deterministic checks to calibrated semantic judges, golden regression suites, visual briefing, and rigorous statistical deployment gating.\n\n```\n[1. Deterministic Guardrails] -> [2. Anchored LLM Judge] -> [3. Human Calibration] -> [4. Golden Regression Suite] -> [5. The Visual Brief & Mandatory Checkpoint Gate]\n```\n\nSee [CARD.md](CARD.md) for the companion summary card, 5-stage reference matrix, and quality checklist.\nConsult [agent-harness-architect](..\u002Fagent-harness-architect\u002FSKILL.md) for runtime loop containment and [crafting-skills](..\u002Fcrafting-skills\u002FSKILL.md) for skill authoring standards.\n\n---\n\n## 1. Deterministic Guardrail Configuration (Layer 1)\n\nDeploy zero-cost, sub-millisecond rule-based sanity checks on 100% of LLM outputs before calling downstream evaluators or users. Layer 1 catches approximately 30% of defective outputs for $0 in API costs.\n\n1. **Schema & Syntax Validation**:\n   - Verify output validity when structured output is expected (e.g. `json.loads()`, YAML, XML, SQL).\n   - If invalid, immediately trigger in-flight self-repair or return a structural failure.\n2. **Length Boundary Verification**:\n   - Enforce minimum and maximum character\u002Ftoken bounds (e.g., $[10, 5000]$ chars).\n   - Catch truncated generations, infinite token loops, or blank\u002Fwhitespace responses.\n3. **Hallucinated Link Detection**:\n   - Extract URLs using regex (`https?:\u002F\u002F[^\\s)[\\]{}\"'\u003C>]+`).\n   - Match extracted URLs against an authorized domain whitelist or verify DNS resolution. Flag unverified links as hallucinations.\n4. **Refusal Phrase Detection**:\n   - Scan for canned refusal strings (*\"as an ai\"*, *\"i cannot\"*, *\"i'm unable to\"*, *\"i don't have access\"*).\n   - Flag unprompted refusals on benign operational tasks.\n5. **Domain-Specific Linters**:\n   - Attach fast domain linters (e.g. Python AST syntax check, SQL query syntax parsing, email greeting\u002Fsignature verification).\n\n> **Completion criterion**: `DeterministicEvaluator` deployed with schema, length, link, refusal, and domain checks running synchronously in \u003C1ms with zero LLM API calls.\n\n---\n\n## 2. Anchored Rubric & LLM Judge Synthesis (Layer 2)\n\nAutomate semantic evaluation using an isolated LLM judge (e.g. `gpt-4o-mini`, `gemini-1.5-flash`) scoring responses against concrete, observable rubrics.\n\n1. **Concrete Observable Anchor Rubrics**:\n   - Never ask a judge to \"rate this 1 to 10\" without definitions.\n   - Anchor each integer on a 1-to-5 scale to observable structural facts:\n     - **Score 1**: Completely irrelevant, incorrect, harmful, or unresponsive.\n     - **Score 2**: On-topic but contains major factual errors, hallucinated concepts, or critical omissions.\n     - **Score 3**: Partially correct; core points present but misses important nuances or structure.\n     - **Score 4**: Fully accurate and helpful; directly answers prompt with only minor aesthetic issues.\n     - **Score 5**: Comprehensive, flawless accuracy, well-structured, and directly addresses the user's intent.\n2. **Constrained JSON Judge Protocol**:\n   - Set temperature to 0.0 to eliminate judge creativity.\n   - Mandate strict JSON schema output: `{\"score\": \u003Cint 1-5>, \"reasoning\": \"\u003C2-3 sentences>\"}`.\n   - Bake the complete rubric text directly into the system prompt for every evaluation call.\n3. **Multi-Judge Median Consensus**:\n   - For sensitive regression gates or high-variance prompts, execute 3 independent judge calls.\n   - Take the median score (`int(np.median(scores))`) to neutralize stochastic outlier runs.\n\n> **Completion criterion**: `LLMJudge` configured with observable 1-5 anchors, temperature 0.0, strict JSON schema output, and optional 3-judge median consensus.\n\n---\n\n## 3. Ground Truth Human Calibration (Layer 3)\n\nAutomated LLM judges have blind spots around tone, domain jargon, and superficial correctness. Layer 3 maintains alignment between automated scoring and human judgment.\n\n1. **Lightweight Annotation Collector**:\n   - Deploy a minimalist CLI or web annotation tool presenting input question and model response to human evaluators.\n   - Record score (1-5), annotator ID, and optional qualitative feedback to append-only JSONL files (`annotations.jsonl`).\n2. **Inter-Annotator Agreement (Cohen's Kappa)**:\n   - Measure statistical agreement between human reviewers corrected for chance:\n     $$\\kappa = \\frac{P_o - P_e}{1 - P_e}$$\n   - **Gating Threshold**: Assert $\\kappa > 0.6$. If $\\kappa \\le 0.6$, the rubric definitions are ambiguous; halt evaluation and refine rubric anchors with concrete examples.\n3. **Automated Judge Drift Audit**:\n   - Maintain a benchmark calibration set of 20-30 human-annotated examples.\n   - Re-run the automated LLM judge monthly against this set.\n   - If mean absolute deviation between judge scores and human scores exceeds 0.5 points, trigger rubric recalibration.\n\n> **Completion criterion**: Human annotation collection active in JSONL format, Cohen's Kappa verified $> 0.6$, and monthly judge drift tracking established.\n\n---\n\n## 4. Golden Dataset & CI\u002FCD Regression Suite\n\nPrevent prompt changes, model updates, or retrieval modifications from introducing silent regressions across previously working edge cases.\n\n1. **Curate Representative Golden Dataset**:\n   - Build a version-controlled dataset of 50 to 100 realistic test examples (`golden_dataset.json`).\n   - Balance across domain categories and difficulty tiers (\"easy\", \"medium\", \"hard\").\n   - Include edge cases that previously triggered production failures.\n2. **Incident Ingestion Flywheel**:\n   - Establish a zero-drop rule: every production incident or user bug report must be converted into a permanent golden test case.\n3. **Automated Regression Pipeline**:\n   - Execute Layer 1 deterministic checks and Layer 2 LLM judge across all golden examples.\n   - Assert zero Layer 1 failures and average composite score above threshold (e.g. $\\ge 3.5$).\n   - Record run metadata (run ID, timestamp, model version, prompt hash, pass rate, failure details) in versioned audit logs.\n\n> **Completion criterion**: Golden dataset populated with 50-100 balanced cases, CI\u002FCD regression pipeline executing on every prompt\u002Fmodel change, and incident ingestion active.\n\n---\n\n## 5. The Visual Brief & Mandatory Checkpoint Gate\n\nSynthesize evaluation results into an interactive visual dashboard and enforce an unyielding statistical release gate before deploying prompt modifications, model migrations, or retrieval changes to production.\n\n### The Visual Brief\n\nGenerate a standalone interactive HTML visual brief in `%TEMP%\u002Fself-eval-brief-\u003Ctimestamp>.html` summarizing the regression run:\n1. **Visual Telemetry**:\n   - Render a dark-themed (`#0d1117`) dashboard loading Tailwind CSS and Mermaid.js via CDN.\n   - Embed a **Score Distribution & Regression Delta** chart comparing pre-change vs post-change scores across all golden categories.\n   - Render a **Layer 1 Failure Analysis Matrix** detailing any deterministic catches.\n   - Display the **Judge Reasoning Transcript** for all examples scoring below threshold ($\u003C 3.5$).\n2. **Delivery**:\n   - Output the clickable absolute file path to the user\u002Foperator for rapid visual inspection.\n\n### Mandatory Checkpoint Gate\n\nEnforce an automated approval gate with `RequestFeedback: true` in CI\u002FCD before any production deployment:\n1. **Statistical Significance Testing (Paired Student's t-Test)**:\n   - Compute paired $t$-test statistic and $p$-value using `scipy.stats.ttest_rel(scores_after, scores_before)` on identical golden test cases.\n   - **Gating Invariant**:\n     $$p \u003C 0.05 \\quad \\text{AND} \\quad (\\bar{X}_{\\text{after}} - \\bar{X}_{\\text{before}}) > 0$$\n2. **Gating Decision Rule**:\n   - **PASS**: $p \u003C 0.05$ and Mean Difference $> 0 \\implies$ Release approved.\n   - **BLOCK (Noise)**: $p \\ge 0.05 \\implies$ Release blocked; improvement is indistinguishable from random noise.\n   - **BLOCK (Regression)**: Mean Difference $\\le 0 \\implies$ Release blocked; system has regressed.\n\n> **Completion criterion**: Interactive HTML visual brief generated in `%TEMP%`, and deployment blocked until statistical significance ($p \u003C 0.05$) passes the Mandatory Checkpoint Gate.\n\n---\n\n## Anti-Patterns\n\n- **Big-Bang Eval Syndrome** — Delaying evaluation until complex multi-judge and human rating platforms are built instead of deploying Layer 1 deterministic checks on Day 1.\n- **Vibe-Based Judging** — Asking an LLM judge to \"rate this from 1 to 10\" without discrete, observable anchor definitions for each score.\n- **Fragile Output Parsing** — Allowing judge models to return free-form prose and attempting regex scraping rather than mandating strict JSON schema output.\n- **The Expert Blindspot** — Relying exclusively on automated LLM judges without human calibration loops, missing toxic tone, domain jargon, or superficial correctness.\n- **Regression Blindness** — Tweaking a prompt to resolve one visible user defect without re-running the complete golden dataset to verify adjacent capabilities.\n- **Noise Chasing** — Deploying prompt modifications or temperature adjustments based on small average score increases without paired $t$-test statistical significance ($p \u003C 0.05$).","# Self-Evaluating AI Pipeline: Automated LLM Quality Architecture\n\nThe `self-evaluating-ai-pipeline` skill provides the definitive engineering framework for evaluating non-deterministic Large Language Model applications. Traditional software unit tests (`assert output == expected`) fail because LLM outputs vary across runs. This skill implements an opinionated, production-grade 3-layer evaluation funnel, regression suites with golden datasets, and statistical significance testing.\n\nSynthesized from Jude Otine's foundational literature (*How to Build a Self-Evaluating AI System*, freeCodeCamp, 2026), this pipeline enforces the **Shu-Ha-Ri framework principle**: moving from rigid deterministic checks to calibrated semantic judges, golden regression suites, visual briefing, and rigorous statistical deployment gating.\n\n```\n[1. Deterministic Guardrails] -> [2. Anchored LLM Judge] -> [3. Human Calibration] -> [4. Golden Regression Suite] -> [5. The Visual Brief & Mandatory Checkpoint Gate]\n```\n\nSee [CARD.md](CARD.md) for the companion summary card, 5-stage reference matrix, and quality checklist.\nConsult [agent-harness-architect](..\u002Fagent-harness-architect\u002FSKILL.md) for runtime loop containment and [crafting-skills](..\u002Fcrafting-skills\u002FSKILL.md) for skill authoring standards.\n\n---\n\n## 1. Deterministic Guardrail Configuration (Layer 1)\n\nDeploy zero-cost, sub-millisecond rule-based sanity checks on 100% of LLM outputs before calling downstream evaluators or users. Layer 1 catches approximately 30% of defective outputs for $0 in API costs.\n\n1. **Schema & Syntax Validation**:\n   - Verify output validity when structured output is expected (e.g. `json.loads()`, YAML, XML, SQL).\n   - If invalid, immediately trigger in-flight self-repair or return a structural failure.\n2. **Length Boundary Verification**:\n   - Enforce minimum and maximum character\u002Ftoken bounds (e.g., $[10, 5000]$ chars).\n   - Catch truncated generations, infinite token loops, or blank\u002Fwhitespace responses.\n3. **Hallucinated Link Detection**:\n   - Extract URLs using regex (`https?:\u002F\u002F[^\\s)[\\]{}\"'\u003C>]+`).\n   - Match extracted URLs against an authorized domain whitelist or verify DNS resolution. Flag unverified links as hallucinations.\n4. **Refusal Phrase Detection**:\n   - Scan for canned refusal strings (*\"as an ai\"*, *\"i cannot\"*, *\"i'm unable to\"*, *\"i don't have access\"*).\n   - Flag unprompted refusals on benign operational tasks.\n5. **Domain-Specific Linters**:\n   - Attach fast domain linters (e.g. Python AST syntax check, SQL query syntax parsing, email greeting\u002Fsignature verification).\n\n> **Completion criterion**: `DeterministicEvaluator` deployed with schema, length, link, refusal, and domain checks running synchronously in \u003C1ms with zero LLM API calls.\n\n---\n\n## 2. Anchored Rubric & LLM Judge Synthesis (Layer 2)\n\nAutomate semantic evaluation using an isolated LLM judge (e.g. `gpt-4o-mini`, `gemini-1.5-flash`) scoring responses against concrete, observable rubrics.\n\n1. **Concrete Observable Anchor Rubrics**:\n   - Never ask a judge to \"rate this 1 to 10\" without definitions.\n   - Anchor each integer on a 1-to-5 scale to observable structural facts:\n     - **Score 1**: Completely irrelevant, incorrect, harmful, or unresponsive.\n     - **Score 2**: On-topic but contains major factual errors, hallucinated concepts, or critical omissions.\n     - **Score 3**: Partially correct; core points present but misses important nuances or structure.\n     - **Score 4**: Fully accurate and helpful; directly answers prompt with only minor aesthetic issues.\n     - **Score 5**: Comprehensive, flawless accuracy, well-structured, and directly addresses the user's intent.\n2. **Constrained JSON Judge Protocol**:\n   - Set temperature to 0.0 to eliminate judge creativity.\n   - Mandate strict JSON schema output: `{\"score\": \u003Cint 1-5>, \"reasoning\": \"\u003C2-3 sentences>\"}`.\n   - Bake the complete rubric text directly into the system prompt for every evaluation call.\n3. **Multi-Judge Median Consensus**:\n   - For sensitive regression gates or high-variance prompts, execute 3 independent judge calls.\n   - Take the median score (`int(np.median(scores))`) to neutralize stochastic outlier runs.\n\n> **Completion criterion**: `LLMJudge` configured with observable 1-5 anchors, temperature 0.0, strict JSON schema output, and optional 3-judge median consensus.\n\n---\n\n## 3. Ground Truth Human Calibration (Layer 3)\n\nAutomated LLM judges have blind spots around tone, domain jargon, and superficial correctness. Layer 3 maintains alignment between automated scoring and human judgment.\n\n1. **Lightweight Annotation Collector**:\n   - Deploy a minimalist CLI or web annotation tool presenting input question and model response to human evaluators.\n   - Record score (1-5), annotator ID, and optional qualitative feedback to append-only JSONL files (`annotations.jsonl`).\n2. **Inter-Annotator Agreement (Cohen's Kappa)**:\n   - Measure statistical agreement between human reviewers corrected for chance:\n     $$\\kappa = \\frac{P_o - P_e}{1 - P_e}$$\n   - **Gating Threshold**: Assert $\\kappa > 0.6$. If $\\kappa \\le 0.6$, the rubric definitions are ambiguous; halt evaluation and refine rubric anchors with concrete examples.\n3. **Automated Judge Drift Audit**:\n   - Maintain a benchmark calibration set of 20-30 human-annotated examples.\n   - Re-run the automated LLM judge monthly against this set.\n   - If mean absolute deviation between judge scores and human scores exceeds 0.5 points, trigger rubric recalibration.\n\n> **Completion criterion**: Human annotation collection active in JSONL format, Cohen's Kappa verified $> 0.6$, and monthly judge drift tracking established.\n\n---\n\n## 4. Golden Dataset & CI\u002FCD Regression Suite\n\nPrevent prompt changes, model updates, or retrieval modifications from introducing silent regressions across previously working edge cases.\n\n1. **Curate Representative Golden Dataset**:\n   - Build a version-controlled dataset of 50 to 100 realistic test examples (`golden_dataset.json`).\n   - Balance across domain categories and difficulty tiers (\"easy\", \"medium\", \"hard\").\n   - Include edge cases that previously triggered production failures.\n2. **Incident Ingestion Flywheel**:\n   - Establish a zero-drop rule: every production incident or user bug report must be converted into a permanent golden test case.\n3. **Automated Regression Pipeline**:\n   - Execute Layer 1 deterministic checks and Layer 2 LLM judge across all golden examples.\n   - Assert zero Layer 1 failures and average composite score above threshold (e.g. $\\ge 3.5$).\n   - Record run metadata (run ID, timestamp, model version, prompt hash, pass rate, failure details) in versioned audit logs.\n\n> **Completion criterion**: Golden dataset populated with 50-100 balanced cases, CI\u002FCD regression pipeline executing on every prompt\u002Fmodel change, and incident ingestion active.\n\n---\n\n## 5. The Visual Brief & Mandatory Checkpoint Gate\n\nSynthesize evaluation results into an interactive visual dashboard and enforce an unyielding statistical release gate before deploying prompt modifications, model migrations, or retrieval changes to production.\n\n### The Visual Brief\n\nGenerate a standalone interactive HTML visual brief in `%TEMP%\u002Fself-eval-brief-\u003Ctimestamp>.html` summarizing the regression run:\n1. **Visual Telemetry**:\n   - Render a dark-themed (`#0d1117`) dashboard loading Tailwind CSS and Mermaid.js via CDN.\n   - Embed a **Score Distribution & Regression Delta** chart comparing pre-change vs post-change scores across all golden categories.\n   - Render a **Layer 1 Failure Analysis Matrix** detailing any deterministic catches.\n   - Display the **Judge Reasoning Transcript** for all examples scoring below threshold ($\u003C 3.5$).\n2. **Delivery**:\n   - Output the clickable absolute file path to the user\u002Foperator for rapid visual inspection.\n\n### Mandatory Checkpoint Gate\n\nEnforce an automated approval gate with `RequestFeedback: true` in CI\u002FCD before any production deployment:\n1. **Statistical Significance Testing (Paired Student's t-Test)**:\n   - Compute paired $t$-test statistic and $p$-value using `scipy.stats.ttest_rel(scores_after, scores_before)` on identical golden test cases.\n   - **Gating Invariant**:\n     $$p \u003C 0.05 \\quad \\text{AND} \\quad (\\bar{X}_{\\text{after}} - \\bar{X}_{\\text{before}}) > 0$$\n2. **Gating Decision Rule**:\n   - **PASS**: $p \u003C 0.05$ and Mean Difference $> 0 \\implies$ Release approved.\n   - **BLOCK (Noise)**: $p \\ge 0.05 \\implies$ Release blocked; improvement is indistinguishable from random noise.\n   - **BLOCK (Regression)**: Mean Difference $\\le 0 \\implies$ Release blocked; system has regressed.\n\n> **Completion criterion**: Interactive HTML visual brief generated in `%TEMP%`, and deployment blocked until statistical significance ($p \u003C 0.05$) passes the Mandatory Checkpoint Gate.\n\n---\n\n## Anti-Patterns\n\n- **Big-Bang Eval Syndrome** — Delaying evaluation until complex multi-judge and human rating platforms are built instead of deploying Layer 1 deterministic checks on Day 1.\n- **Vibe-Based Judging** — Asking an LLM judge to \"rate this from 1 to 10\" without discrete, observable anchor definitions for each score.\n- **Fragile Output Parsing** — Allowing judge models to return free-form prose and attempting regex scraping rather than mandating strict JSON schema output.\n- **The Expert Blindspot** — Relying exclusively on automated LLM judges without human calibration loops, missing toxic tone, domain jargon, or superficial correctness.\n- **Regression Blindness** — Tweaking a prompt to resolve one visible user defect without re-running the complete golden dataset to verify adjacent capabilities.\n- **Noise Chasing** — Deploying prompt modifications or temperature adjustments based on small average score increases without paired $t$-test statistical significance ($p \u003C 0.05$).",{"name":5,"description":7},"main","eff3b00e36d1d233ad4a4389605c82fbfa0326d3",".agents\u002Fskills\u002Fself-evaluating-ai-pipeline","https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness","https:\u002F\u002Fgithub.com\u002FSpectreDeath","I'm here to manage stress so I wouldn't expect earth shattering break throughs I'm just an enthusiast that likes making stuff. ","2026-11-15 05:32:58",{"source":8,"skillName":5,"commands":32,"prompt":42,"zipAvailable":43},[33,36,39],{"tool":34,"command":35},"npx","npx skills add SpectreDeath\u002FBrain-Harness --skill self-evaluating-ai-pipeline",{"tool":37,"command":38},"bunx","bunx skills add SpectreDeath\u002FBrain-Harness --skill self-evaluating-ai-pipeline",{"tool":40,"command":41},"pnpm","pnpm dlx skills add SpectreDeath\u002FBrain-Harness --skill self-evaluating-ai-pipeline","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fself-evaluating-ai-pipeline\u002FSKILL.md",true,[45,51,55,59,65],{"path":46,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":49,"sourceHtmlUrl":50},"CARD.md","REFERENCE","参考文档",3474,"https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fself-evaluating-ai-pipeline\u002FCARD.md",{"path":52,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":53,"sourceHtmlUrl":54},"config.default.yaml",629,"https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fself-evaluating-ai-pipeline\u002Fconfig.default.yaml",{"path":56,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":57,"sourceHtmlUrl":58},"resources\u002Fgolden_dataset.json",12051,"https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fself-evaluating-ai-pipeline\u002Fresources\u002Fgolden_dataset.json",{"path":60,"resourceType":61,"resourceTypeDesc":62,"sizeBytes":63,"sourceHtmlUrl":64},"scripts\u002Fself_eval_cli.py","SCRIPT","脚本",4900,"https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fself-evaluating-ai-pipeline\u002Fscripts\u002Fself_eval_cli.py",{"path":66,"resourceType":61,"resourceTypeDesc":62,"sizeBytes":67,"sourceHtmlUrl":68},"scripts\u002Fself_evaluating_pipeline_engine.py",32397,"https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fself-evaluating-ai-pipeline\u002Fscripts\u002Fself_evaluating_pipeline_engine.py",[70,77,84,91,98,105],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":19},"01M7E0W6FR68J2N9CXCZTAQJFX","ui-behavior-primitive-architect","Architect, audit, and verify zero-dependency headless UI behavior primitives, WebMCP agent bridges, WAI-ARIA\u002FSSR\u002FRTL tri-modal checks, and framework expert routing. Do not use for static CSS theme styling or backend database migrations.","https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fui-behavior-primitive-architect\u002FSKILL.md",[],[],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":19},"01M7E0VZ7JQNW34G271DENFGYP","trustgraph-ontology-governor","Govern knowledge graphs, SHACL-AF triple policies, OntologyRAG subclass validation, PROV-O dual named graphs, and Laya entropy decision scoring. Do not use for relational SQL queries or generic web scraping.","https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Ftrustgraph-ontology-governor\u002FSKILL.md",[],[],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":19},"01M7E0VQ6Z7R45Q8QAZ4MT39EK","shim-harness-governor","Govern universal CLI agent loops, 4-tier tool permissions, ancestor-symlink workspace sandboxes, deterministic-first command routing, collision-safe MCP namespacing, and layered skill loading. Do not use for static UI styling or external SQL analytics.","https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fshim-harness-governor\u002FSKILL.md",[],[],{"skillNo":92,"name":93,"nameZh":6,"description":94,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":19},"01M7E0VC18NMP1QBSRJ3TPQ7MB","graphiti-zep-memory-governor","Govern bi-temporal knowledge graphs with Graphiti future-edge clamping, MinHash\u002FLSH entity dedup, RRF\u002FMMR reranking, and Zep typed ontologies. Do not use for static SQL joins or generic web crawling.","https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fgraphiti-zep-memory-governor\u002FSKILL.md",[],[],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":19},"01M7E0V12ABE1RW3ZRCST8VH5Z","agora-mcp-workbench-architect","Architect, audit, and orchestrate Agora Workbench MCP servers, loopback sidecars, unified state graphs, gateway\u002Fdispatcher connectors, and policy-enforced data lakes. Do not use for static frontend CSS styling or non-MCP desktop GUI automation.","https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fagora-mcp-workbench-architect\u002FSKILL.md",[],[],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":19},"01M7BFYG6QT3484VC57TZ19V7P","skill-eval-assurance-runner","Evaluate and assure agent skills using small-N Wilson confidence intervals, unbiased pass^k reliability, deterministic trajectory graders, preflight boundary honesty, and sealed digest verification. Do not use for subjective prose review or unconstrained production deployment.","https:\u002F\u002Fgithub.com\u002FSpectreDeath\u002FBrain-Harness\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fskill-eval-assurance-runner\u002FSKILL.md",[],[]]