[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7ASV6N6CDJXEVRKX4FAZF3B":3,"skill-same:01M7ASV6N6CDJXEVRKX4FAZF3B":45},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":23,"commitSha":24,"skillDirectory":25,"forksCount":26,"repositoryHtmlUrl":27,"repositoryHomepageUrl":6,"ownerHtmlUrl":28,"ownerBlogUrl":29,"ownerDescription":6,"updatedAt":30,"install":31,"resources":44},"01M7ASV6N6CDJXEVRKX4FAZF3B","promptfoo-evals",null,"Design and run promptfoo evaluations for LLM behavior regressions or comparisons across prompts, agent rules, and models. Use for repeatable behavioral cases with explicit assertions, not ordinary unit tests or static configuration checks.","saski\u002Farnesto","saski","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F140356?v=4","01M7ASBDSNXTRYDYB4K1PEA1RV","01M7ASBDTAKBDAAV99S1GE7HK6",6,"Unlicense","https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fpromptfoo-evals\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: promptfoo-evals\ndescription: Design and run promptfoo evaluations for LLM behavior regressions or comparisons across prompts, agent rules, and models. Use for repeatable behavioral cases with explicit assertions, not ordinary unit tests or static configuration checks.\n---\n\n# Promptfoo Evaluations\n\nUse promptfoo when the question concerns observable LLM behavior across a\nrepeatable set of inputs. A deterministic function or configuration invariant\nusually belongs in the project's ordinary test suite instead.\n\n## Define what is being evaluated\n\nState the behavior, available evidence, and unacceptable outcome before writing\nthe case. Choose the evaluation boundary explicitly:\n\n- **Prompt only:** send the candidate prompt and controlled context to a model.\n- **Configured client:** invoke the real agent entry point with its instructions,\n  tools, and runtime configuration. Attribute the result to that combination.\n\nA client passing a case does not prove the harness caused the behavior. To test\nthat claim, compare with and without the relevant instructions while keeping\nthe cases, model, and runtime settings stable. Preserve a user's requested\nprovider; report unavailable models rather than silently replacing them.\n\n## Build a useful case and grader\n\nStart with a representative failure mode from actual use. Include enough\nevidence to determine the expected behavior without putting the expected answer\nin the evaluated prompt. Add ambiguity or pressure only when it models a real\nrisk. Expand coverage from observed gaps, not arbitrary case counts.\n\nPrefer deterministic assertions for structure, exact facts, and prohibited\nactions. Check the substantive claim as well as the format: valid JSON alone\nis not success. Avoid exact prose matching when equivalent wording is valid.\nFor tool-use claims, inspect execution traces rather than the agent's self-report.\n\nExercise the grader with a known correct response and plausible incorrect\nresponses before trusting it. Use an LLM judge only for properties that need\nsemantic judgment; record its model and rubric and check its judgments against\nexamples reviewed by a person. Do not weaken assertions just to obtain green runs.\n\n## Run and interpret\n\nUse the project's existing installation, configuration, and execution targets.\nFor new integrations, pin the evaluator and commit the dependency lockfile.\nCheck current official documentation when adding unfamiliar provider or\nassertion configuration; avoid copying guessed API syntax.\n\nKeep live provider calls separate from fast deterministic checks. Respect the\ntask's data and spending boundaries. Record model, relevant runtime settings,\ncase revision, and cache use so comparisons can be interpreted. Use fresh\nresponses when claiming a new live verification. For variable behavior, report\nrepeated runs and their counts rather than treating one pass as reliability.\n\nReport behavioral failures separately from provider errors, timeouts, missing\ncredentials, and empty responses. Unavailable providers are not passes, and\nretries must not conceal earlier failures. Retain local reports for inspection;\ndo not commit credentials, private prompts, or runtime databases.\n\n## Arnesto starting point\n\nWhen working in the Arnesto checkout, inspect `evals\u002Fpromptfoo\u002Fpromptfooconfig.yaml`\nand `docs\u002Fdevelopment-guide.md` before changing the evaluation. The existing\ncase checks honest reporting of unexecuted validation through the Codex client.\n\n```sh\nmake install-promptfoo\nmake eval-promptfoo\n```\n\nThese targets are Arnesto-specific; discover equivalent commands in other\nrepositories. The initial case and provider are examples, not a requirement\nto use Codex, structured JSON, or a particular model for every evaluation.\n","# Promptfoo Evaluations\n\nUse promptfoo when the question concerns observable LLM behavior across a\nrepeatable set of inputs. A deterministic function or configuration invariant\nusually belongs in the project's ordinary test suite instead.\n\n## Define what is being evaluated\n\nState the behavior, available evidence, and unacceptable outcome before writing\nthe case. Choose the evaluation boundary explicitly:\n\n- **Prompt only:** send the candidate prompt and controlled context to a model.\n- **Configured client:** invoke the real agent entry point with its instructions,\n  tools, and runtime configuration. Attribute the result to that combination.\n\nA client passing a case does not prove the harness caused the behavior. To test\nthat claim, compare with and without the relevant instructions while keeping\nthe cases, model, and runtime settings stable. Preserve a user's requested\nprovider; report unavailable models rather than silently replacing them.\n\n## Build a useful case and grader\n\nStart with a representative failure mode from actual use. Include enough\nevidence to determine the expected behavior without putting the expected answer\nin the evaluated prompt. Add ambiguity or pressure only when it models a real\nrisk. Expand coverage from observed gaps, not arbitrary case counts.\n\nPrefer deterministic assertions for structure, exact facts, and prohibited\nactions. Check the substantive claim as well as the format: valid JSON alone\nis not success. Avoid exact prose matching when equivalent wording is valid.\nFor tool-use claims, inspect execution traces rather than the agent's self-report.\n\nExercise the grader with a known correct response and plausible incorrect\nresponses before trusting it. Use an LLM judge only for properties that need\nsemantic judgment; record its model and rubric and check its judgments against\nexamples reviewed by a person. Do not weaken assertions just to obtain green runs.\n\n## Run and interpret\n\nUse the project's existing installation, configuration, and execution targets.\nFor new integrations, pin the evaluator and commit the dependency lockfile.\nCheck current official documentation when adding unfamiliar provider or\nassertion configuration; avoid copying guessed API syntax.\n\nKeep live provider calls separate from fast deterministic checks. Respect the\ntask's data and spending boundaries. Record model, relevant runtime settings,\ncase revision, and cache use so comparisons can be interpreted. Use fresh\nresponses when claiming a new live verification. For variable behavior, report\nrepeated runs and their counts rather than treating one pass as reliability.\n\nReport behavioral failures separately from provider errors, timeouts, missing\ncredentials, and empty responses. Unavailable providers are not passes, and\nretries must not conceal earlier failures. Retain local reports for inspection;\ndo not commit credentials, private prompts, or runtime databases.\n\n## Arnesto starting point\n\nWhen working in the Arnesto checkout, inspect `evals\u002Fpromptfoo\u002Fpromptfooconfig.yaml`\nand `docs\u002Fdevelopment-guide.md` before changing the evaluation. The existing\ncase checks honest reporting of unexecuted validation through the Codex client.\n\n```sh\nmake install-promptfoo\nmake eval-promptfoo\n```\n\nThese targets are Arnesto-specific; discover equivalent commands in other\nrepositories. The initial case and provider are examples, not a requirement\nto use Codex, structured JSON, or a particular model for every evaluation.\n",{"name":5,"description":7},"main","3c24fc2c035029b08c7326774bee3fef8fb95566",".agents\u002Fskills\u002Fpromptfoo-evals",0,"https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto","https:\u002F\u002Fgithub.com\u002Fsaski","http:\u002F\u002Fsaski.com","2026-11-12 20:09:16",{"source":8,"skillName":5,"commands":32,"prompt":42,"zipAvailable":43},[33,36,39],{"tool":34,"command":35},"npx","npx skills add saski\u002Farnesto --skill promptfoo-evals",{"tool":37,"command":38},"bunx","bunx skills add saski\u002Farnesto --skill promptfoo-evals",{"tool":40,"command":41},"pnpm","pnpm dlx skills add saski\u002Farnesto --skill promptfoo-evals","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fpromptfoo-evals\u002FSKILL.md",true,[],[46,53,60,67,74,81],{"skillNo":47,"name":48,"nameZh":6,"description":49,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":50,"categories":51,"keywords":52,"publishedAt":18},"01M7AT7F5FSHNAMFMF01N0QA1X","visualize","Use \u002Fvisualize for creating compact charts, diagrams, or other visuals inline as part of the conversation itself. It chooses among Markdown tables, Mermaid, inline HTML, and other supported in-transcript forms. Use Canvas for durable artifacts outside the transcript that users may revisit, refine, or share.","https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto\u002Fblob\u002Fmain\u002F.cursor\u002Fskills-cursor\u002Fvisualize\u002FSKILL.md",[],[],{"skillNo":54,"name":55,"nameZh":6,"description":56,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":57,"categories":58,"keywords":59,"publishedAt":18},"01M7AT79MHSAR3JDDMP7DGERM9","update-cursor-settings","Modify Cursor\u002FVSCode user settings in settings.json. Use when you want to change editor settings, preferences, configuration, themes, font size, tab size, format on save, auto save, keybindings, or any settings.json values.","https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto\u002Fblob\u002Fmain\u002F.cursor\u002Fskills-cursor\u002Fupdate-cursor-settings\u002FSKILL.md",[],[],{"skillNo":61,"name":62,"nameZh":6,"description":63,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":64,"categories":65,"keywords":66,"publishedAt":18},"01M7AT73WF3Y22E0FKYAMCCJES","update-cli-config","View and modify Cursor CLI configuration settings in ~\u002F.cursor\u002Fcli-config.json. Use when the user wants to change CLI settings, configure permissions, switch approval mode, enable vim mode, toggle display options, configure sandbox, or manage any CLI preferences.","https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto\u002Fblob\u002Fmain\u002F.cursor\u002Fskills-cursor\u002Fupdate-cli-config\u002FSKILL.md",[],[],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":18},"01M7AT6F3FBP85AJJ5BJNR8B5W","statusline","Configure a custom status line in the CLI. Use when the user mentions status line, statusline, statusLine, CLI status bar, prompt footer customization, or wants to add session context above the prompt.","https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto\u002Fblob\u002Fmain\u002F.cursor\u002Fskills-cursor\u002Fstatusline\u002FSKILL.md",[],[],{"skillNo":75,"name":76,"nameZh":6,"description":77,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":18},"01M7AT6D24QFS5XRAFN8S5XJ7W","split-to-prs","Split current work into small reviewable PRs. Use when the user asks to split a chat, set of changes, branch, or PR.","https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto\u002Fblob\u002Fmain\u002F.cursor\u002Fskills-cursor\u002Fsplit-to-prs\u002FSKILL.md",[],[],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":87,"publishedAt":18},"01M7AT69FYFN84XZKW452JV5DF","shell","Runs the rest of a \u002Fshell request as a literal shell command. Use only when the user explicitly invokes \u002Fshell and wants the following text executed directly in the terminal.","https:\u002F\u002Fgithub.com\u002Fsaski\u002Farnesto\u002Fblob\u002Fmain\u002F.cursor\u002Fskills-cursor\u002Fshell\u002FSKILL.md",[],[]]