[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7F1V6ZPE61AE9HMSS9YT7PS":3,"skill-same:01M7F1V6ZPE61AE9HMSS9YT7PS":67},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":28,"commitSha":29,"skillDirectory":30,"forksCount":31,"repositoryHtmlUrl":32,"repositoryHomepageUrl":6,"ownerHtmlUrl":33,"ownerBlogUrl":6,"ownerDescription":34,"updatedAt":35,"install":36,"resources":48},"01M7F1V6ZPE61AE9HMSS9YT7PS","benchmark-evaluation",null,"Run a bounded ChestAgentBench or Eurorad multimodal evaluation through an OpenAI-compatible chat client, with local or URL-backed images, manifest checks, JSONL logging, interruption handling, and cautious result interpretation.","lza6\u002FArmor-piercing-system-prompts","lza6","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F128880206?v=4","01M7EB3K7WPX5T28EJ2NVHE6EG","Chinese-tingfeng","01M7EB3K97E1KQ16QWKJMFNGX8",11,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F05-技能与代理开发\u002Fmedrax\u002Fsub-skills\u002Fbenchmark-evaluation\u002FSKILL.md",[],[],"2026-11-16 10:05:34",false,"---\nname: benchmark-evaluation\ndescription: \"Run a bounded ChestAgentBench or Eurorad multimodal evaluation\n  through an OpenAI-compatible chat client, with local or URL-backed images,\n  manifest checks, JSONL logging, interruption handling, and cautious result\n  interpretation.\"\ndisable-model-invocation: true\nmetadata:\n  disco-role: operating\nlicense: Apache 2.0\n---\n\n# Benchmark evaluation\n\nUse this skill to evaluate a vision-capable chat model on an already prepared,\nlocal ChestAgentBench-style manifest or Eurorad-derived case set. This is an\n**evaluation runner**, not a dataset builder and not a clinical decision tool.\nRoute dataset scraping, question generation, and exploratory experiments to\ntheir sibling skills. Route MedRAX agent\u002Ftool initialization to the agent\u002Ftool\nruntime skill.\n\n## Safety and scope gate\n\n- Start with a bounded plan: select the manifest, image mode, model, output\n  prefix, and a small `--max-cases` value. Treat one case as the smoke run and\n  expand only after its log and image resolution are sound.\n- Network access, credentials, API spend, provider-side inference, and URL image\n  downloads are deferred until the user explicitly authorizes the run. A\n  manifest check never needs an API key or network access.\n- ChestAgentBench\u002FEurorad accuracy is a benchmark observation, not clinical\n  validation, patient-specific advice, or evidence of deployment safety.\n- Do not place API keys, bearer headers, inline base64 image payloads, or\n  unredacted prompts in a report or shared log. Preserve only the redacted\n  JSONL contract described in [data-formats.md](references\u002Fdata-formats.md).\n\n## Quickstart route\n\n1. Confirm that the supplied benchmark runner exposes the flags documented in\n   [cli-reference.md](references\u002Fcli-reference.md). Do not silently substitute\n   a dataset-generation command.\n2. Validate a local JSONL manifest and image root before any client creation:\n   ```bash\n   python scripts\u002Fvalidate_case_manifest.py cases\u002Fmetadata.jsonl --root cases\n   ```\n   Nested image arrays are supported. Missing local images are errors; a case\n   with no image references is reported as a skip candidate. Use `--help` to\n   inspect all bounded-check options.\n3. For a local-image smoke run, after authorization and local data readiness:\n   ```bash\n   python quickstart.py --model \"$MODEL\" --temperature 0.2 \\\n     --log-prefix smoke-local --max-cases 1\n   ```\n   The runner loads a JSONL benchmark split, creates an OpenAI-compatible\n   client, embeds local images as data URLs, and logs one JSON object per case.\n4. For URL-backed images, plan exactly one case first:\n   ```bash\n   python quickstart.py --model \"$MODEL\" --temperature 0.2 \\\n     --log-prefix smoke-url --max-cases 1 --use-urls\n   ```\n   URL mode still fetches and base64-embeds the images before the API call; it\n   is therefore both network work and inference work. Stop if the URL fetch is\n   not explicitly authorized or cannot be bounded.\n5. On success, inspect the redacted log for `question_id`, `status`, answer,\n   duration, usage, and image-source counts before increasing the case limit.\n   Never infer benchmark quality from a single case.\n\n## Prompt and answer contract\n\nThe multimodal request contains a medical-case preamble, the case's multiple\nchoice `question`, and an instruction to answer only from the supplied images\nand case information. The system role asks for only one choice letter among\n`A\u002FB\u002FC\u002FD\u002FE\u002FF`. The expected answer is the manifest's `answer` field. Preserve\nthe model's raw answer for audit, but normalize only a separate analysis field;\ndo not rewrite the source answer or silently score prose as a letter.\n\nA valid case should have a stable `question_id`, non-empty `question`, an\nexpected `answer`, and `images` for local mode or `image_source_urls` for URL\nmode. Image fields may be a string, a list, or nested lists. Flatten them in\norder and preserve that order in the log. Empty or missing images do not call\nthe model: record `status: \"skipped\"` and `reason: \"no_images\"`.\n\n## Configuration and bounded execution\n\n- `--model` is the effective request model and takes precedence over the\n  runner's default. `OPENAI_MODEL` is a useful provider\u002Fconfiguration\n  convention, but do not assume the runner reads it automatically; pass its\n  value explicitly when needed.\n- `OPENAI_API_KEY` is required by the reference quickstart. `OPENAI_BASE_URL`,\n  when set, is passed to the OpenAI-compatible client. Use a provider's exact\n  compatible endpoint and model name; a custom base URL does not make an\n  incompatible model multimodal.\n- Keep temperature low and record it. `--max-cases` limits the selected prefix\n  of the dataset; it is not a random sample and is not a confidence interval.\n- Use a writable, non-sensitive log prefix. The timestamped output filename is\n  derived from the prefix and current time; check that it exists and is\n  non-empty after the run.\n- The reference request retries transient-looking failures up to three times\n  with exponential waits (approximately 4–10 seconds between attempts). A\n  final exception is logged and raised, so persistent provider errors can stop\n  the loop rather than being silently counted as wrong answers.\n- SIGINT\u002FSIGTERM sets a shutdown event checked between cases. Expect the\n  current request to finish or fail; interruption is not a promise to cancel a\n  provider request. Keep the log as a partial run and record processed\u002Fskipped\n  counts.\n\n## Interpretation and handoff\n\nSeparate these quantities: attempted cases, successful model responses,\nno-image skips, request errors, valid-choice answers, and exact answer matches.\nReport the manifest version\u002Fsource description, image mode, model\u002Fbase-url\nclass (not secrets), temperature, case limit, interruption state, and log\ncompleteness. Do not call skipped cases incorrect unless the evaluation plan\nexplicitly defines that policy. A benchmark score is sensitive to image\navailability, prompt wording, model\u002Fversion, provider behavior, and case\nordering; compare runs only when those controls are held constant.\n\nFor detailed schemas, workflows, flags, and predictable failures, read:\n\n- [workflows.md](references\u002Fworkflows.md)\n- [data-formats.md](references\u002Fdata-formats.md)\n- [cli-reference.md](references\u002Fcli-reference.md)\n- [troubleshooting.md](references\u002Ftroubleshooting.md)\n","# Benchmark evaluation\n\nUse this skill to evaluate a vision-capable chat model on an already prepared,\nlocal ChestAgentBench-style manifest or Eurorad-derived case set. This is an\n**evaluation runner**, not a dataset builder and not a clinical decision tool.\nRoute dataset scraping, question generation, and exploratory experiments to\ntheir sibling skills. Route MedRAX agent\u002Ftool initialization to the agent\u002Ftool\nruntime skill.\n\n## Safety and scope gate\n\n- Start with a bounded plan: select the manifest, image mode, model, output\n  prefix, and a small `--max-cases` value. Treat one case as the smoke run and\n  expand only after its log and image resolution are sound.\n- Network access, credentials, API spend, provider-side inference, and URL image\n  downloads are deferred until the user explicitly authorizes the run. A\n  manifest check never needs an API key or network access.\n- ChestAgentBench\u002FEurorad accuracy is a benchmark observation, not clinical\n  validation, patient-specific advice, or evidence of deployment safety.\n- Do not place API keys, bearer headers, inline base64 image payloads, or\n  unredacted prompts in a report or shared log. Preserve only the redacted\n  JSONL contract described in [data-formats.md](references\u002Fdata-formats.md).\n\n## Quickstart route\n\n1. Confirm that the supplied benchmark runner exposes the flags documented in\n   [cli-reference.md](references\u002Fcli-reference.md). Do not silently substitute\n   a dataset-generation command.\n2. Validate a local JSONL manifest and image root before any client creation:\n   ```bash\n   python scripts\u002Fvalidate_case_manifest.py cases\u002Fmetadata.jsonl --root cases\n   ```\n   Nested image arrays are supported. Missing local images are errors; a case\n   with no image references is reported as a skip candidate. Use `--help` to\n   inspect all bounded-check options.\n3. For a local-image smoke run, after authorization and local data readiness:\n   ```bash\n   python quickstart.py --model \"$MODEL\" --temperature 0.2 \\\n     --log-prefix smoke-local --max-cases 1\n   ```\n   The runner loads a JSONL benchmark split, creates an OpenAI-compatible\n   client, embeds local images as data URLs, and logs one JSON object per case.\n4. For URL-backed images, plan exactly one case first:\n   ```bash\n   python quickstart.py --model \"$MODEL\" --temperature 0.2 \\\n     --log-prefix smoke-url --max-cases 1 --use-urls\n   ```\n   URL mode still fetches and base64-embeds the images before the API call; it\n   is therefore both network work and inference work. Stop if the URL fetch is\n   not explicitly authorized or cannot be bounded.\n5. On success, inspect the redacted log for `question_id`, `status`, answer,\n   duration, usage, and image-source counts before increasing the case limit.\n   Never infer benchmark quality from a single case.\n\n## Prompt and answer contract\n\nThe multimodal request contains a medical-case preamble, the case's multiple\nchoice `question`, and an instruction to answer only from the supplied images\nand case information. The system role asks for only one choice letter among\n`A\u002FB\u002FC\u002FD\u002FE\u002FF`. The expected answer is the manifest's `answer` field. Preserve\nthe model's raw answer for audit, but normalize only a separate analysis field;\ndo not rewrite the source answer or silently score prose as a letter.\n\nA valid case should have a stable `question_id`, non-empty `question`, an\nexpected `answer`, and `images` for local mode or `image_source_urls` for URL\nmode. Image fields may be a string, a list, or nested lists. Flatten them in\norder and preserve that order in the log. Empty or missing images do not call\nthe model: record `status: \"skipped\"` and `reason: \"no_images\"`.\n\n## Configuration and bounded execution\n\n- `--model` is the effective request model and takes precedence over the\n  runner's default. `OPENAI_MODEL` is a useful provider\u002Fconfiguration\n  convention, but do not assume the runner reads it automatically; pass its\n  value explicitly when needed.\n- `OPENAI_API_KEY` is required by the reference quickstart. `OPENAI_BASE_URL`,\n  when set, is passed to the OpenAI-compatible client. Use a provider's exact\n  compatible endpoint and model name; a custom base URL does not make an\n  incompatible model multimodal.\n- Keep temperature low and record it. `--max-cases` limits the selected prefix\n  of the dataset; it is not a random sample and is not a confidence interval.\n- Use a writable, non-sensitive log prefix. The timestamped output filename is\n  derived from the prefix and current time; check that it exists and is\n  non-empty after the run.\n- The reference request retries transient-looking failures up to three times\n  with exponential waits (approximately 4–10 seconds between attempts). A\n  final exception is logged and raised, so persistent provider errors can stop\n  the loop rather than being silently counted as wrong answers.\n- SIGINT\u002FSIGTERM sets a shutdown event checked between cases. Expect the\n  current request to finish or fail; interruption is not a promise to cancel a\n  provider request. Keep the log as a partial run and record processed\u002Fskipped\n  counts.\n\n## Interpretation and handoff\n\nSeparate these quantities: attempted cases, successful model responses,\nno-image skips, request errors, valid-choice answers, and exact answer matches.\nReport the manifest version\u002Fsource description, image mode, model\u002Fbase-url\nclass (not secrets), temperature, case limit, interruption state, and log\ncompleteness. Do not call skipped cases incorrect unless the evaluation plan\nexplicitly defines that policy. A benchmark score is sensitive to image\navailability, prompt wording, model\u002Fversion, provider behavior, and case\nordering; compare runs only when those controls are held constant.\n\nFor detailed schemas, workflows, flags, and predictable failures, read:\n\n- [workflows.md](references\u002Fworkflows.md)\n- [data-formats.md](references\u002Fdata-formats.md)\n- [cli-reference.md](references\u002Fcli-reference.md)\n- [troubleshooting.md](references\u002Ftroubleshooting.md)\n",{"name":5,"license":24,"metadata":25,"description":7,"disable-model-invocation":27},"Apache 2.0",{"disco-role":26},"operating",true,"main","d69096e239f2a89aabc2cd078c47664a9d3b1e11","05-技能与代理开发\u002Fmedrax\u002Fsub-skills\u002Fbenchmark-evaluation",3,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts","https:\u002F\u002Fgithub.com\u002Flza6","个人vx：Tf00798\r\n个人✈：@TFGY999","2026-10-08 14:29:22",{"source":8,"skillName":5,"commands":37,"prompt":47,"zipAvailable":27},[38,41,44],{"tool":39,"command":40},"npx","npx skills add lza6\u002FArmor-piercing-system-prompts --skill benchmark-evaluation",{"tool":42,"command":43},"bunx","bunx skills add lza6\u002FArmor-piercing-system-prompts --skill benchmark-evaluation",{"tool":45,"command":46},"pnpm","pnpm dlx skills add lza6\u002FArmor-piercing-system-prompts --skill benchmark-evaluation","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F05-技能与代理开发\u002Fmedrax\u002Fsub-skills\u002Fbenchmark-evaluation\u002FSKILL.md",[49,55,59,63],{"path":50,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":53,"sourceHtmlUrl":54},"references\u002Fcli-reference.md","REFERENCE","参考文档",2722,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F05-技能与代理开发\u002Fmedrax\u002Fsub-skills\u002Fbenchmark-evaluation\u002Freferences\u002Fcli-reference.md",{"path":56,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":57,"sourceHtmlUrl":58},"references\u002Fdata-formats.md",3540,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F05-技能与代理开发\u002Fmedrax\u002Fsub-skills\u002Fbenchmark-evaluation\u002Freferences\u002Fdata-formats.md",{"path":60,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":61,"sourceHtmlUrl":62},"references\u002Ftroubleshooting.md",3619,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F05-技能与代理开发\u002Fmedrax\u002Fsub-skills\u002Fbenchmark-evaluation\u002Freferences\u002Ftroubleshooting.md",{"path":64,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":65,"sourceHtmlUrl":66},"references\u002Fworkflows.md",3802,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F05-技能与代理开发\u002Fmedrax\u002Fsub-skills\u002Fbenchmark-evaluation\u002Freferences\u002Fworkflows.md",[68,76,81,88,95,102],{"skillNo":69,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":72,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":19},"01M7FFG485KY83TD6HSC4YPCMC","obsidian-clipper-template-creator","Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.","MIT","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobsidian-clipper-template-creator\u002FSKILL.md",[],[],{"skillNo":77,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":72,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":19},"01M7FFG0T6AZ0STEJPSB3CX84K","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobsidian-clipper-template-creator-antigravity-awesome-skills-main\u002FSKILL.md",[],[],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":72,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":87,"publishedAt":19},"01M7FFFY0QEM14SKWPDYGMXM2P","python-api","Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP\u002FmAP semantics, and source-style import diagnostics.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002Fsub-skills\u002Fpython-api\u002FSKILL.md",[],[],{"skillNo":89,"name":90,"nameZh":6,"description":91,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":72,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":94,"publishedAt":19},"01M7FFFVZQZ2GJ0C0TGP7JHD20","file-evaluation","Folder-based PASCAL VOC AP\u002FmAP evaluation from ground-truth and detection text files.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002Fsub-skills\u002Ffile-evaluation\u002FSKILL.md",[],[],{"skillNo":96,"name":97,"nameZh":6,"description":98,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":72,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":101,"publishedAt":19},"01M7FFFSFNYSE77SJ6PE1CK2DA","object-detection-metrics","Guides legacy Object-Detection-Metrics PASCAL VOC AP\u002FmAP evaluation from detection text folders and direct Python API classes.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002FSKILL.md",[],[],{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":72,"licenseFileUrl":6,"sourceHtmlUrl":106,"categories":107,"keywords":108,"publishedAt":19},"01M7FFFP0GCK1RVXBCPQ8APBMT","nx-workspace-patterns","Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fnx-workspace-patterns\u002FSKILL.md",[],[]]