[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6H0WEW8ZF5DFQ8CE34ZNSVG":3,"skill-same:01M6H0WEW8ZF5DFQ8CE34ZNSVG":77},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":16,"sourceHtmlUrl":17,"categories":18,"keywords":19,"publishedAt":35,"linkOnly":36,"markdownContent":37,"markdownBody":38,"frontMatter":39,"refName":40,"commitSha":41,"skillDirectory":42,"forksCount":43,"repositoryHtmlUrl":44,"repositoryHomepageUrl":6,"ownerHtmlUrl":45,"ownerBlogUrl":46,"ownerDescription":47,"updatedAt":48,"install":49,"resources":62},"01M6H0WEW8ZF5DFQ8CE34ZNSVG","verify-answer-claims-against-chunks",null,"Use when a generated answer may state things the retrieved chunks never said. Checks every claim sentence against its cited chunk and flags or drops unsupported ones before shipping.","jiayaoqijia\u002Fcryptoskill","jiayaoqijia","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F1453879?v=4","01M6GMYTX7Q7S504VC6E756ENR","YQ","01M6GMYV1QKJ1CC75MTTGT9PR0",81,"MIT","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-verify-answer-claims-against-chunks\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-verify-answer-claims-against-chunks\u002FSKILL.md",[],[20,21,22,23,24,25,26,27,28,29,30,31,32,33,34],"verify","answer","claims","verify answer claims against chunks","chunks","shipping","state","chunk","generated","use","never","flags","claim","things","when","2026-11-10 10:12:30",false,"---\nname: verify-answer-claims-against-chunks\ndescription: Use when a generated answer may state things the retrieved chunks never said. Checks every claim sentence against its cited chunk and flags or drops unsupported ones before shipping.\n---\n\n# Verify Answer Claims Against Chunks\n\nRAG does not guarantee faithfulness: the model can blend retrieved facts with memory. Verification is mechanical — each claim must be entailed by a chunk it cites, or it does not ship.\n\n## Procedure\n\n1. Split the answer into atomic claims (one fact per sentence or clause) so support can be checked per claim.\n2. Require a citation marker per claim and map it to the chunk id (`attach-stable-citation-ids`).\n3. For each claim, check entailment against its cited chunk: an NLI model (`facebook\u002Fbart-large-mnli`) or a strict judge prompt returning `supported` \u002F `unsupported` \u002F `contradicted`.\n4. Drop or flag any `unsupported` claim; treat `contradicted` as a hard failure that blocks the answer.\n5. Flag claims whose number, date or entity does not appear verbatim in the cited chunk — numeric drift is the common fabrication.\n6. Distinguish `unsupported` (not in the chunk) from `contradicted` (chunk says otherwise); they need different fixes.\n7. Sample the verifier's own decisions by hand — a lenient NLI model will pass fabrications.\n8. Report the faithfulness rate (supported claims \u002F total) as a metric that must not fall when generation changes.\n9. Keep the claim-to-chunk map with the answer, so a reader can see the support for each line.\n10. Re-run the check on any prompt or model change; faithfulness is a property of the pair, not the model alone.\n\n11. Store the verifier output per claim with the answer, so a rejection can be reviewed rather than re-run.\n\n## Pitfalls\n\n- Checking the whole answer against the whole context instead of claim-by-claim, hiding one unsupported line in a supported paragraph.\n- A verifier that reads the claim and the chunk in one prompt and defers to the claim's confidence.\n- Treating a paraphrase with a wrong number as supported because the rest of the sentence matches.\n- Dropping the citation and keeping the claim, so an unsupported fact ships unlabelled.\n- Using the same model for generation and verification with no adversarial check on the verifier.\n- Ignoring `contradicted` claims because they are rarer than `unsupported` ones.\n\n- A verifier prompt that asks 'is this consistent' and accepts topical overlap as entailment.\n- Checking only claims that carry a citation, missing uncited claims added by the model.\n\n## Verification\n\n    python3 faithfulness.py --answer answer.md --context chunks.jsonl\n    # passes when every claim resolves to a chunk and the contradicted count is 0\n\nReport to the user: claims checked, supported\u002Funsupported\u002Fcontradicted counts, faithfulness rate, and every dropped claim with its reason.\n","# Verify Answer Claims Against Chunks\n\nRAG does not guarantee faithfulness: the model can blend retrieved facts with memory. Verification is mechanical — each claim must be entailed by a chunk it cites, or it does not ship.\n\n## Procedure\n\n1. Split the answer into atomic claims (one fact per sentence or clause) so support can be checked per claim.\n2. Require a citation marker per claim and map it to the chunk id (`attach-stable-citation-ids`).\n3. For each claim, check entailment against its cited chunk: an NLI model (`facebook\u002Fbart-large-mnli`) or a strict judge prompt returning `supported` \u002F `unsupported` \u002F `contradicted`.\n4. Drop or flag any `unsupported` claim; treat `contradicted` as a hard failure that blocks the answer.\n5. Flag claims whose number, date or entity does not appear verbatim in the cited chunk — numeric drift is the common fabrication.\n6. Distinguish `unsupported` (not in the chunk) from `contradicted` (chunk says otherwise); they need different fixes.\n7. Sample the verifier's own decisions by hand — a lenient NLI model will pass fabrications.\n8. Report the faithfulness rate (supported claims \u002F total) as a metric that must not fall when generation changes.\n9. Keep the claim-to-chunk map with the answer, so a reader can see the support for each line.\n10. Re-run the check on any prompt or model change; faithfulness is a property of the pair, not the model alone.\n\n11. Store the verifier output per claim with the answer, so a rejection can be reviewed rather than re-run.\n\n## Pitfalls\n\n- Checking the whole answer against the whole context instead of claim-by-claim, hiding one unsupported line in a supported paragraph.\n- A verifier that reads the claim and the chunk in one prompt and defers to the claim's confidence.\n- Treating a paraphrase with a wrong number as supported because the rest of the sentence matches.\n- Dropping the citation and keeping the claim, so an unsupported fact ships unlabelled.\n- Using the same model for generation and verification with no adversarial check on the verifier.\n- Ignoring `contradicted` claims because they are rarer than `unsupported` ones.\n\n- A verifier prompt that asks 'is this consistent' and accepts topical overlap as entailment.\n- Checking only claims that carry a citation, missing uncited claims added by the model.\n\n## Verification\n\n    python3 faithfulness.py --answer answer.md --context chunks.jsonl\n    # passes when every claim resolves to a chunk and the contradicted count is 0\n\nReport to the user: claims checked, supported\u002Funsupported\u002Fcontradicted counts, faithfulness rate, and every dropped claim with its reason.\n",{"name":5,"description":7},"main","3df1d0d67b30ff15236561f8c0f9f06d33f42dfd","skills\u002Fpayments\u002F0xklen-verify-answer-claims-against-chunks",17,"https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia","https:\u002F\u002Feth2030.com","Hardcore infra. Building @alt-research and applied cryptography.","2026-11-13 02:31:19",{"source":8,"skillName":5,"commands":50,"prompt":60,"zipAvailable":61},[51,54,57],{"tool":52,"command":53},"npx","npx skills add jiayaoqijia\u002Fcryptoskill --skill verify-answer-claims-against-chunks",{"tool":55,"command":56},"bunx","bunx skills add jiayaoqijia\u002Fcryptoskill --skill verify-answer-claims-against-chunks",{"tool":58,"command":59},"pnpm","pnpm dlx skills add jiayaoqijia\u002Fcryptoskill --skill verify-answer-claims-against-chunks","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-verify-answer-claims-against-chunks\u002FSKILL.md",true,[63,67,73],{"path":64,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":66,"sourceHtmlUrl":16},"LICENSE","许可证",1063,{"path":68,"resourceType":69,"resourceTypeDesc":70,"sizeBytes":71,"sourceHtmlUrl":72},"SOURCE.md","REFERENCE","参考文档",361,"https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-verify-answer-claims-against-chunks\u002FSOURCE.md",{"path":74,"resourceType":69,"resourceTypeDesc":70,"sizeBytes":75,"sourceHtmlUrl":76},"TRUST.auto.yaml",2217,"https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-verify-answer-claims-against-chunks\u002FTRUST.auto.yaml",[78,106,129,147,160,170],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":82,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":35},"01M712C436EQYE43342Q4E7GG1","recipe-performance","Diagnose a specific MetaMask user flow's real-app performance and Sentry critical-user-flow telemetry with deterministic mm-harness recipes. Use for CUF boundaries, cold\u002Fwarm\u002Fbackground-resume behavior, latency regressions, lifecycle tags, duplicate or blocking network requests, long tasks, layout\u002Fpaint cost, screen redraws, React rerenders, or browser trace analysis where tests alone cannot prove the real app.","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fwallets\u002Fmetamask-official-recipe-performance\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fwallets\u002Fmetamask-official-recipe-performance\u002FSKILL.md",[],[86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105],"performance","recipe performance","recipe","mm","sentry","harness","app","deterministic","recipes","flow","cold","critical","analysis","tasks","react","browser","layout","screen","network","metamask",{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":110,"licenseFileUrl":111,"sourceHtmlUrl":112,"categories":113,"keywords":114,"publishedAt":35},"01M70XT2F8A5CQZ81JBP98KSW3","mexc-spot-websocket","MEXC Spot WebSocket workflows for public streams, protobuf handling, depth maintenance, listen keys, and private streams without exposing API keys.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fexchanges\u002Fmncrftfrcnm-mexc-spot-websocket\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fexchanges\u002Fmncrftfrcnm-mexc-spot-websocket\u002FSKILL.md",[],[115,116,117,118,119,120,121,122,123,124,125,126,127,128],"mexc spot websocket","spot","websocket","api","workflows","public","listen","streams","depth","keys","for","api keys","maintenance","protobuf",{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":110,"licenseFileUrl":133,"sourceHtmlUrl":134,"categories":135,"keywords":136,"publishedAt":35},"01M70XT0MBXN3WZY1BKN0CPNKR","mexc-spot-rest","MEXC Spot REST workflows for public data, signed account requests, orders, wallet\u002Fadmin actions, and secure signing without exposing API keys.","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fexchanges\u002Fmncrftfrcnm-mexc-spot-rest\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fexchanges\u002Fmncrftfrcnm-mexc-spot-rest\u002FSKILL.md",[],[137,116,138,139,140,141,142,126,125,124,143,144,145,120,146,119,118],"mexc spot rest","REST","admin","signing","Wallet","requests","actions","secure","data","orders",{"skillNo":148,"name":149,"nameZh":6,"description":150,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":110,"licenseFileUrl":151,"sourceHtmlUrl":152,"categories":153,"keywords":154,"publishedAt":35},"01M70XSYSVZ2N0N91XCAR2CAK0","mexc-futures-websocket","MEXC Futures WebSocket workflows for public streams, private account\u002Forder\u002Fposition streams, ping\u002Fpong, subscriptions, and depth maintenance.","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fexchanges\u002Fmncrftfrcnm-mexc-futures-websocket\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fexchanges\u002Fmncrftfrcnm-mexc-futures-websocket\u002FSKILL.md",[],[155,156,117,157,158,127,125,159,123,122,120,119],"mexc futures websocket","futures","position","ping","subscriptions",{"skillNo":161,"name":162,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":163,"sourceHtmlUrl":164,"categories":165,"keywords":166,"publishedAt":35},"01M6H0ZWHVE5W04WG2DKQE2SQC","0xklen-version-and-refresh-sanctions-lists","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-version-and-refresh-sanctions-lists\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-version-and-refresh-sanctions-lists\u002FSKILL.md",[],[167,168,169],"refresh","0xklen version and refresh sanctions lists","version",{"skillNo":171,"name":172,"nameZh":6,"description":173,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":174,"sourceHtmlUrl":175,"categories":176,"keywords":177,"publishedAt":35},"01M6H0Z28Q3R8J0S799GJQWXM3","verify-tests-cover-changed-lines","Use when a PR adds code that the existing suite may not exercise. Measures patch coverage — the share of newly changed lines hit by tests — before approving.","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-verify-tests-cover-changed-lines\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fjiayaoqijia\u002Fcryptoskill\u002Fblob\u002Fmain\u002Fskills\u002Fpayments\u002F0xklen-verify-tests-cover-changed-lines\u002FSKILL.md",[],[178,179,180,20,181,182,183,29,184,185,186],"tests","cover","verify tests cover changed lines","coverage","pr","exercise","patch","code","share"]