[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M69BGH0NBCAMZET5QWNKTJ83":3,"skill-same:01M69BGH0NBCAMZET5QWNKTJ83":53},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":23,"linkOnly":24,"markdownContent":25,"markdownBody":26,"frontMatter":27,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":33,"repositoryHtmlUrl":34,"repositoryHomepageUrl":6,"ownerHtmlUrl":35,"ownerBlogUrl":36,"ownerDescription":37,"updatedAt":38,"install":39,"resources":52},"01M69BGH0NBCAMZET5QWNKTJ83","context-intelligence-evaluation-methodology",null,"Use when deciding how to measure a context-intelligence tool signal. Covers metric design across quality\u002Fefficiency\u002Fefficacy axes, artifact-metric avoidance via precursor measurement, A\u002FB and statistical-N discipline, and test-data fidelity.","microsoft\u002Famplifier-bundle-context-intelligence","microsoft","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F6154722?v=4","01M698VYDXZ6JW5WH3CRV650R8","Microsoft","01M69BGBVCCWXG1MNNF1E8TCWN",5,"MIT","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence\u002Fblob\u002Fmain\u002Fskills\u002Fcontext-intelligence-evaluation-methodology\u002FSKILL.md",[],[19,20,21,22],"context intelligence evaluation methodology","design","test","reasoning","2026-11-10 10:12:30",false,"---\nname: context-intelligence-evaluation-methodology\nversion: 1.0.0\ndescription: Use when deciding how to measure a context-intelligence tool signal. Covers metric design across quality\u002Fefficiency\u002Fefficacy axes, artifact-metric avoidance via precursor measurement, A\u002FB and statistical-N discipline, and test-data fidelity.\nuser-invocable: false\nallowed-tools: read_file, glob, grep, delegate, load_skill, todo\nmodel_role: reasoning\nlicense: MIT\n---\n\n# Context Intelligence Evaluation Methodology\n\nMode-only skill for *how to measure*. It complements — and never restates —\n`context-intelligence-eval-design` (which owns scenario mechanics and the two-layer\nstructural\u002Fbehavioral structure) and `digital-twin-universe` (which owns the DTU machinery).\n\n## Scope\n\n### In scope\n\n- **Metric design.** Choose metrics across three axes: **quality** (did it detect what the user\n  means?), **efficiency** (token\u002Ftool cost to detect), **efficacy** (does detection drive the\n  right outcome?).\n- **Measure the precursor, not only the failure.** Prefer leading indicators (e.g. bounded vs\n  climbing context growth) over lagging ones (e.g. a final timeout). The precursor is testable\n  in a short window; the full failure often is not.\n- **A\u002FB + statistical-N discipline.** A single green run is not proof of a behavioral change.\n  Compare a control arm against a treatment arm; require N independent trials and report the\n  pass rate, not a single anecdote.\n- **Test data fidelity.** Validate on **real sessions** where available, or on **faithfully\n  modeled synthetic** corpora that preserve the distribution that matters (e.g. heavy-tail file\n  sizes), never on data hand-shaped to pass.\n\n### Out of scope (point elsewhere — do not restate)\n\n- **Scenario mechanics, the two-layer structure, DTU profile templates \u002F Gitea URL rewrite** →\n  `context-intelligence-eval-design`.\n- **DTU lifecycle, launch\u002Fexec\u002Fprofiles** → the `digital-twin-universe` skill.\n- **\"DTU-as-default\" rationale and the \"artifact-as-success\" anti-pattern** → already authoritative\n  in `context-intelligence-eval-design` and\n  `context-intelligence:context\u002Fcontext-intelligence-primitives-reference.md`. Reference them;\n  do not repeat them.\n\n## Method\n\n1. **Name the user-meaningful outcome** the metric must reflect (from `domain-concepts.md`),\n   not what a detector happens to emit.\n2. **Pick the cheapest axis that discriminates.** If a deterministic efficiency signal (token\n   growth, tool-call count) separates good from bad behavior, prefer it over an LLM-judged\n   quality rubric.\n3. **Identify the precursor.** Ask: what climbs *before* the failure? Measure that in a bounded\n   window.\n4. **Design the A\u002FB.** Control = current behavior; treatment = the change. Hold everything else\n   identical. Decide N and the pass threshold *before* running.\n5. **Choose the corpus.** Real sessions if available; otherwise synthetic that preserves the\n   load-bearing distribution. State the fidelity assumption explicitly.\n\n> **Apply (do not restate) the event-semantics principle:** when a metric depends on what an\n> event *means*, consult the authoritative ecosystem expert agents for that event's semantics —\n> the principle is named once in\n> `context-intelligence:context\u002Fcontext-intelligence-strategy.md`. Do not restate it here.\n","# Context Intelligence Evaluation Methodology\n\nMode-only skill for *how to measure*. It complements — and never restates —\n`context-intelligence-eval-design` (which owns scenario mechanics and the two-layer\nstructural\u002Fbehavioral structure) and `digital-twin-universe` (which owns the DTU machinery).\n\n## Scope\n\n### In scope\n\n- **Metric design.** Choose metrics across three axes: **quality** (did it detect what the user\n  means?), **efficiency** (token\u002Ftool cost to detect), **efficacy** (does detection drive the\n  right outcome?).\n- **Measure the precursor, not only the failure.** Prefer leading indicators (e.g. bounded vs\n  climbing context growth) over lagging ones (e.g. a final timeout). The precursor is testable\n  in a short window; the full failure often is not.\n- **A\u002FB + statistical-N discipline.** A single green run is not proof of a behavioral change.\n  Compare a control arm against a treatment arm; require N independent trials and report the\n  pass rate, not a single anecdote.\n- **Test data fidelity.** Validate on **real sessions** where available, or on **faithfully\n  modeled synthetic** corpora that preserve the distribution that matters (e.g. heavy-tail file\n  sizes), never on data hand-shaped to pass.\n\n### Out of scope (point elsewhere — do not restate)\n\n- **Scenario mechanics, the two-layer structure, DTU profile templates \u002F Gitea URL rewrite** →\n  `context-intelligence-eval-design`.\n- **DTU lifecycle, launch\u002Fexec\u002Fprofiles** → the `digital-twin-universe` skill.\n- **\"DTU-as-default\" rationale and the \"artifact-as-success\" anti-pattern** → already authoritative\n  in `context-intelligence-eval-design` and\n  `context-intelligence:context\u002Fcontext-intelligence-primitives-reference.md`. Reference them;\n  do not repeat them.\n\n## Method\n\n1. **Name the user-meaningful outcome** the metric must reflect (from `domain-concepts.md`),\n   not what a detector happens to emit.\n2. **Pick the cheapest axis that discriminates.** If a deterministic efficiency signal (token\n   growth, tool-call count) separates good from bad behavior, prefer it over an LLM-judged\n   quality rubric.\n3. **Identify the precursor.** Ask: what climbs *before* the failure? Measure that in a bounded\n   window.\n4. **Design the A\u002FB.** Control = current behavior; treatment = the change. Hold everything else\n   identical. Decide N and the pass threshold *before* running.\n5. **Choose the corpus.** Real sessions if available; otherwise synthetic that preserves the\n   load-bearing distribution. State the fidelity assumption explicitly.\n\n> **Apply (do not restate) the event-semantics principle:** when a metric depends on what an\n> event *means*, consult the authoritative ecosystem expert agents for that event's semantics —\n> the principle is named once in\n> `context-intelligence:context\u002Fcontext-intelligence-strategy.md`. Do not restate it here.\n",{"name":5,"license":15,"version":28,"model_role":22,"description":7,"allowed-tools":29,"user-invocable":24},"1.0.0","read_file, glob, grep, delegate, load_skill, todo","main","2a8d7bd1e022971c395ec38ed797bf75fcf7aff7","skills\u002Fcontext-intelligence-evaluation-methodology",7,"https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence","https:\u002F\u002Fgithub.com\u002Fmicrosoft","https:\u002F\u002Fopensource.microsoft.com","Open source projects and samples from Microsoft","2026-11-07 04:52:31",{"source":8,"skillName":5,"commands":40,"prompt":50,"zipAvailable":51},[41,44,47],{"tool":42,"command":43},"npx","npx skills add microsoft\u002Famplifier-bundle-context-intelligence --skill context-intelligence-evaluation-methodology",{"tool":45,"command":46},"bunx","bunx skills add microsoft\u002Famplifier-bundle-context-intelligence --skill context-intelligence-evaluation-methodology",{"tool":48,"command":49},"pnpm","pnpm dlx skills add microsoft\u002Famplifier-bundle-context-intelligence --skill context-intelligence-evaluation-methodology","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence\u002Fblob\u002Fmain\u002Fskills\u002Fcontext-intelligence-evaluation-methodology\u002FSKILL.md",true,[],[54,70,79,95,103,111],{"skillNo":55,"name":56,"nameZh":6,"description":57,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":58,"categories":59,"keywords":65,"publishedAt":23},"01M69BGZVPJBKSZV5N0WMVGR3N","workflow-pattern-analysis","Use when asked how a workflow is failing, what separates a successful run from a failing one, which steps are the most common failure points, or what patterns recur across sessions — analyses failure\u002Fsuccess patterns and signals across many runs using context-intelligence session data.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence\u002Fblob\u002Fmain\u002Fskills\u002Fworkflow-pattern-analysis\u002FSKILL.md",[60],{"categoryNo":61,"categoryName":62,"taxonomyType":63,"taxonomyTypeDesc":64},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[66,67,22,68,69],"workflow pattern analysis","workflow","run","sessions",{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":23},"01M69BGYAYC16BRSB0EGMCKMF5","transcript","Retrieve the prior user and assistant conversation verbatim from the current Context Intelligence capture, or named sessions. Use when a user asks to replay, quote, review, or act on a transcript.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence\u002Fblob\u002Fmain\u002Fskills\u002Ftranscript\u002FSKILL.md",[],[72,77,69,78],"assistant","review",{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":93,"publishedAt":23},"01M69BGWQGCEN13DHHAQGTQ609","context-intelligence-tool-design","Use when selecting a detection strategy and implementation primitive for a context-intelligence signal. Classifies signals as deterministic, probabilistic, llm-evaluated, or hybrid, and applies the cheapest-sufficient-capability principle.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence\u002Fblob\u002Fmain\u002Fskills\u002Fcontext-intelligence-tool-design\u002FSKILL.md",[85,88],{"categoryNo":86,"categoryName":87,"taxonomyType":63,"taxonomyTypeDesc":64},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":89,"categoryName":90,"taxonomyType":91,"taxonomyTypeDesc":92},"01JBSEEDCAT000000000000032","设计师","PROFESSION","适用职业",[20,94,22],"context intelligence tool design",{"skillNo":96,"name":97,"nameZh":6,"description":98,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":101,"publishedAt":23},"01M69BGVHJJFCNZMGM21FNWMFS","context-intelligence-session-reconstruction","Use when local Amplifier session files are missing, broken, or incomplete — rebuilds events.jsonl, transcript.jsonl, and metadata.json from the context-intelligence graph server.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence\u002Fblob\u002Fmain\u002Fskills\u002Fcontext-intelligence-session-reconstruction\u002FSKILL.md",[],[102],"context intelligence session reconstruction",{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":23},"01M69BGT14VQD1M3P12WZGTQCS","context-intelligence-session-navigation","Use when extracting session data directly from JSONL files — the fallback when the graph server is unavailable or outside graph-analyst.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence\u002Fblob\u002Fmain\u002Fskills\u002Fcontext-intelligence-session-navigation\u002FSKILL.md",[],[110],"context intelligence session navigation",{"skillNo":112,"name":113,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":115,"categories":116,"keywords":123,"publishedAt":23},"01M69BGRET24BR6PTYANVE57F4","context-intelligence-server-data-ops","Exact step order, wording, and the \"session details\" block format for the delete flows the server-data-ops agent drives — delete the current session, clean up every session from the current working directory, find-then-delete a session by description, and delete a session someone else created.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Famplifier-bundle-context-intelligence\u002Fblob\u002Fmain\u002Fskills\u002Fcontext-intelligence-server-data-ops\u002FSKILL.md",[117,120],{"categoryNo":118,"categoryName":119,"taxonomyType":63,"taxonomyTypeDesc":64},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":121,"categoryName":122,"taxonomyType":91,"taxonomyTypeDesc":92},"01JBSEEDCAT000000000000033","数据分析师",[124,125,126],"context intelligence server data ops","directory","delete"]