[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7829WHPJF7FR6R8647RG89W":3,"skill-same:01M7829WHPJF7FR6R8647RG89W":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":6,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M7829WHPJF7FR6R8647RG89W","adk-evals",null,"Complete reference for writing, running, and iterating on evals (automated conversation tests) for ADK agents. Covers eval file format, all assertion types, CLI usage, and per-primitive testing patterns.","Lord1Egypt\u002Fawesome-skill-forge","Lord1Egypt","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F99746359?v=4","01M7592YDNT0XK9WJ0T2MQVPDN","Mohamed Mounir Ali AL Keshita","01M75GRFNHC8VZNWEB8DKDE433",3,"MIT","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fcommunity\u002Fskills_sh\u002Fs\u002Fskills-sh_botpress_skills_adk-evals\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: adk-evals\ndescription: Complete reference for writing, running, and iterating on evals (automated conversation tests) for ADK agents. Covers eval file format, all assertion types, CLI usage, and per-primitive testing patterns.\nlicense: MIT\n---\n\n# ADK Evals Skill\n\n## What are Evals?\n\nEvals are automated conversation tests for ADK agents. Each eval defines a scenario — a sequence of user messages or events — and asserts on what the bot should do: what it says, which tools it calls, how state changes, which workflows run, and more.\n\nEvals run against a live dev bot (`adk dev`), so they test the full stack — not mocks.\n\n## When to Use This Skill\n\nUse this skill when the developer asks about:\n\n- **Writing evals** — file format, assertions, turn types, setup\n- **Running evals** — CLI commands, filtering, output interpretation\n- **Testing specific primitives** — how to test actions, tools, workflows, conversations, state\n- **The testing loop** — write → run → inspect traces → iterate\n- **CI integration** — exit codes, `--format json` flag, tagging strategies\n- **Eval configuration** — idleTimeout, judgePassThreshold, judgeModel\n\nOr when you are developing an ADK bot and need to write the equivalent of unit\u002Fend-to-end tests.\n\n**Trigger questions:**\n- \"How do I write an eval?\"\n- \"How do I test my workflow?\"\n- \"How do I assert that a tool was called with specific params?\"\n- \"My eval is failing, how do I debug it?\"\n- \"How do I test that the bot stays silent?\"\n- \"How do I run evals in CI?\"\n- \"How do I seed state before an eval?\"\n- \"How do I trigger a workflow in an eval?\"\n\n## Available Documentation\n\n| File | Contents |\n|------|----------|\n| `references\u002Feval-format.md` | Complete file format — all fields, turn types, assertion categories, match operators, setup, outcome, options |\n| `references\u002Ftesting-workflow.md` | Running evals, interpreting output, using traces, the write → test → iterate loop, CI integration |\n| `references\u002Ftest-patterns.md` | Per-primitive patterns for actions, tools, workflows, conversations, and state |\n\n## How to Answer\n\n1. **Writing an eval** → Read `eval-format.md` for structure and assertions\n2. **Running evals** → Read `testing-workflow.md` for CLI commands and output\n3. **Testing a specific primitive** → Read `test-patterns.md` for the relevant section\n4. **Debugging a failure** → Combine `testing-workflow.md` (inspect traces) + `eval-format.md` (check assertion syntax)\n\n---\n\n## Quick Reference\n\n### Eval file structure\n\n```typescript\nimport { Eval } from '@botpress\u002Fevals'\n\nexport default new Eval({\n  name: 'greeting',\n  type: 'regression',\n  tags: ['basic'],\n\n  setup: {\n    state: { bot: { welcomeSent: false } },\n    workflow: { trigger: 'onboarding', input: { userId: 'test-1' } },\n  },\n\n  conversation: [\n    {\n      user: 'Hi!',\n      assert: {\n        response: [\n          { not_contains: 'error' },\n          { llm_judge: 'Response is friendly and offers to help' },\n        ],\n        tools: [{ not_called: 'createTicket' }],\n        state: [{ path: 'conversation.greeted', equals: true }],\n      },\n    },\n  ],\n\n  outcome: {\n    state: [{ path: 'conversation.greeted', equals: true }],\n  },\n\n  options: {\n    idleTimeout: 60000,\n    judgePassThreshold: 4,\n  },\n})\n```\n\n### Turn types\n\n| Turn | When to use |\n|------|------------|\n| `user: 'message'` | Standard user message |\n| `event: { type, payload }` | Non-message trigger (webhook, integration event) |\n| `expectSilence: true` | Assert bot does NOT respond |\n\n### Assertion categories\n\n| Category | What it checks |\n|----------|---------------|\n| `response` | Bot reply text (contains, not_contains, matches, llm_judge) |\n| `tools` | Tool calls (called, not_called, call_order, params) |\n| `state` | Bot\u002Fuser\u002Fconversation state (equals, changed) |\n| `workflow` | Workflow execution (entered, completed) |\n| `timing` | Response time in ms (lte, gte) |\n\n### CLI commands\n\n```bash\nadk evals                        # run all evals\nadk evals \u003Cname>                 # run one eval\nadk evals --tag \u003Ctag>            # filter by tag\nadk evals --type regression      # filter by type\nadk evals --verbose              # show all assertions\nadk evals --format json          # JSON output for CI\n\nadk evals runs                   # list recent runs\nadk evals runs --latest          # most recent run\nadk evals runs --latest -v       # with full details\n```\n\n---\n\n## Critical Patterns\n\n✅ **Every turn needs `user` or `event`**\n\n```typescript\n\u002F\u002F CORRECT\n{ user: 'hello', expectSilence: true }\n{ event: { type: 'payment.failed' }, expectSilence: true }\n```\n\n❌ **`expectSilence` alone is not a valid turn**\n\n```typescript\n\u002F\u002F WRONG — missing user or event\n{ expectSilence: true }\n```\n\n---\n\n✅ **Assert tool params to verify correct extraction**\n\n```typescript\n\u002F\u002F CORRECT — verifies the LLM extracted the right values\n{ called: 'createTicket', params: { priority: { equals: 'high' } } }\n```\n\n❌ **Only asserting the tool was called**\n\n```typescript\n\u002F\u002F INCOMPLETE — doesn't verify params were correct\n{ called: 'createTicket' }\n```\n\n---\n\n✅ **Use `outcome` for post-conversation state and workflow assertions**\n\n```typescript\n\u002F\u002F CORRECT — final state checked once after all turns\noutcome: {\n  state: [{ path: 'conversation.resolved', equals: true }],\n  workflow: [{ name: 'ticketFlow', completed: true }],\n}\n```\n\n---\n\n✅ **Seed state to test conditional behavior without running setup turns**\n\n```typescript\n\u002F\u002F CORRECT — start in a known state\nsetup: {\n  state: {\n    user: { plan: 'pro' },\n    conversation: { phase: 'support' },\n  },\n}\n```\n\n❌ **Using conversation turns to set up state (slow and fragile)**\n\n```typescript\n\u002F\u002F WRONG — depends on the bot correctly processing setup turns\nconversation: [\n  { user: 'I am on the pro plan' },      \u002F\u002F hoping bot sets user.plan\n  { user: 'I need help with billing' },   \u002F\u002F actual test turn\n]\n```\n\n---\n\n## Example Questions\n\n**Writing evals:**\n- \"Write an eval that tests my createTicket tool is called with the right priority\"\n- \"How do I assert that the bot stays silent after an internal event?\"\n- \"How do I test a multi-turn conversation where context is retained?\"\n\n**Running evals:**\n- \"How do I run only regression evals?\"\n- \"How do I see which assertions failed and why?\"\n- \"How do I integrate evals into GitHub Actions?\"\n\n**Debugging:**\n- \"My eval says the tool wasn't called but I think it was — how do I check?\"\n- \"How do I inspect what the bot actually did during an eval?\"\n\n**Per-primitive:**\n- \"How do I test a workflow that uses step.sleep()?\"\n- \"How do I test that state changed from the seeded value?\"\n\n---\n\n## Response Format\n\n**Match depth to the question.**\n\n### Simple questions (\"what assertions are available?\", \"how do I run evals?\")\n\nAnswer directly — show the relevant table or CLI command. Don't generate a full eval file for an informational question.\n\n### Writing an eval\n\n1. Show the complete `new Eval({})` call with realistic field values\n2. Include imports (`import { Eval } from '@botpress\u002Fevals'`)\n3. Briefly explain non-obvious assertions — skip if the assertion is self-explanatory\n4. Suggest the CLI command to run it: `adk evals \u003Cname>`\n\n### Debugging a failing eval\n\n1. Ask for or show the failing assertion (`expected` \u002F `actual` diff)\n2. Suggest opening traces in the Dev Console to see what the bot did\n3. Identify whether the issue is in the eval assertion or the bot's behavior\n","# ADK Evals Skill\n\n## What are Evals?\n\nEvals are automated conversation tests for ADK agents. Each eval defines a scenario — a sequence of user messages or events — and asserts on what the bot should do: what it says, which tools it calls, how state changes, which workflows run, and more.\n\nEvals run against a live dev bot (`adk dev`), so they test the full stack — not mocks.\n\n## When to Use This Skill\n\nUse this skill when the developer asks about:\n\n- **Writing evals** — file format, assertions, turn types, setup\n- **Running evals** — CLI commands, filtering, output interpretation\n- **Testing specific primitives** — how to test actions, tools, workflows, conversations, state\n- **The testing loop** — write → run → inspect traces → iterate\n- **CI integration** — exit codes, `--format json` flag, tagging strategies\n- **Eval configuration** — idleTimeout, judgePassThreshold, judgeModel\n\nOr when you are developing an ADK bot and need to write the equivalent of unit\u002Fend-to-end tests.\n\n**Trigger questions:**\n- \"How do I write an eval?\"\n- \"How do I test my workflow?\"\n- \"How do I assert that a tool was called with specific params?\"\n- \"My eval is failing, how do I debug it?\"\n- \"How do I test that the bot stays silent?\"\n- \"How do I run evals in CI?\"\n- \"How do I seed state before an eval?\"\n- \"How do I trigger a workflow in an eval?\"\n\n## Available Documentation\n\n| File | Contents |\n|------|----------|\n| `references\u002Feval-format.md` | Complete file format — all fields, turn types, assertion categories, match operators, setup, outcome, options |\n| `references\u002Ftesting-workflow.md` | Running evals, interpreting output, using traces, the write → test → iterate loop, CI integration |\n| `references\u002Ftest-patterns.md` | Per-primitive patterns for actions, tools, workflows, conversations, and state |\n\n## How to Answer\n\n1. **Writing an eval** → Read `eval-format.md` for structure and assertions\n2. **Running evals** → Read `testing-workflow.md` for CLI commands and output\n3. **Testing a specific primitive** → Read `test-patterns.md` for the relevant section\n4. **Debugging a failure** → Combine `testing-workflow.md` (inspect traces) + `eval-format.md` (check assertion syntax)\n\n---\n\n## Quick Reference\n\n### Eval file structure\n\n```typescript\nimport { Eval } from '@botpress\u002Fevals'\n\nexport default new Eval({\n  name: 'greeting',\n  type: 'regression',\n  tags: ['basic'],\n\n  setup: {\n    state: { bot: { welcomeSent: false } },\n    workflow: { trigger: 'onboarding', input: { userId: 'test-1' } },\n  },\n\n  conversation: [\n    {\n      user: 'Hi!',\n      assert: {\n        response: [\n          { not_contains: 'error' },\n          { llm_judge: 'Response is friendly and offers to help' },\n        ],\n        tools: [{ not_called: 'createTicket' }],\n        state: [{ path: 'conversation.greeted', equals: true }],\n      },\n    },\n  ],\n\n  outcome: {\n    state: [{ path: 'conversation.greeted', equals: true }],\n  },\n\n  options: {\n    idleTimeout: 60000,\n    judgePassThreshold: 4,\n  },\n})\n```\n\n### Turn types\n\n| Turn | When to use |\n|------|------------|\n| `user: 'message'` | Standard user message |\n| `event: { type, payload }` | Non-message trigger (webhook, integration event) |\n| `expectSilence: true` | Assert bot does NOT respond |\n\n### Assertion categories\n\n| Category | What it checks |\n|----------|---------------|\n| `response` | Bot reply text (contains, not_contains, matches, llm_judge) |\n| `tools` | Tool calls (called, not_called, call_order, params) |\n| `state` | Bot\u002Fuser\u002Fconversation state (equals, changed) |\n| `workflow` | Workflow execution (entered, completed) |\n| `timing` | Response time in ms (lte, gte) |\n\n### CLI commands\n\n```bash\nadk evals                        # run all evals\nadk evals \u003Cname>                 # run one eval\nadk evals --tag \u003Ctag>            # filter by tag\nadk evals --type regression      # filter by type\nadk evals --verbose              # show all assertions\nadk evals --format json          # JSON output for CI\n\nadk evals runs                   # list recent runs\nadk evals runs --latest          # most recent run\nadk evals runs --latest -v       # with full details\n```\n\n---\n\n## Critical Patterns\n\n✅ **Every turn needs `user` or `event`**\n\n```typescript\n\u002F\u002F CORRECT\n{ user: 'hello', expectSilence: true }\n{ event: { type: 'payment.failed' }, expectSilence: true }\n```\n\n❌ **`expectSilence` alone is not a valid turn**\n\n```typescript\n\u002F\u002F WRONG — missing user or event\n{ expectSilence: true }\n```\n\n---\n\n✅ **Assert tool params to verify correct extraction**\n\n```typescript\n\u002F\u002F CORRECT — verifies the LLM extracted the right values\n{ called: 'createTicket', params: { priority: { equals: 'high' } } }\n```\n\n❌ **Only asserting the tool was called**\n\n```typescript\n\u002F\u002F INCOMPLETE — doesn't verify params were correct\n{ called: 'createTicket' }\n```\n\n---\n\n✅ **Use `outcome` for post-conversation state and workflow assertions**\n\n```typescript\n\u002F\u002F CORRECT — final state checked once after all turns\noutcome: {\n  state: [{ path: 'conversation.resolved', equals: true }],\n  workflow: [{ name: 'ticketFlow', completed: true }],\n}\n```\n\n---\n\n✅ **Seed state to test conditional behavior without running setup turns**\n\n```typescript\n\u002F\u002F CORRECT — start in a known state\nsetup: {\n  state: {\n    user: { plan: 'pro' },\n    conversation: { phase: 'support' },\n  },\n}\n```\n\n❌ **Using conversation turns to set up state (slow and fragile)**\n\n```typescript\n\u002F\u002F WRONG — depends on the bot correctly processing setup turns\nconversation: [\n  { user: 'I am on the pro plan' },      \u002F\u002F hoping bot sets user.plan\n  { user: 'I need help with billing' },   \u002F\u002F actual test turn\n]\n```\n\n---\n\n## Example Questions\n\n**Writing evals:**\n- \"Write an eval that tests my createTicket tool is called with the right priority\"\n- \"How do I assert that the bot stays silent after an internal event?\"\n- \"How do I test a multi-turn conversation where context is retained?\"\n\n**Running evals:**\n- \"How do I run only regression evals?\"\n- \"How do I see which assertions failed and why?\"\n- \"How do I integrate evals into GitHub Actions?\"\n\n**Debugging:**\n- \"My eval says the tool wasn't called but I think it was — how do I check?\"\n- \"How do I inspect what the bot actually did during an eval?\"\n\n**Per-primitive:**\n- \"How do I test a workflow that uses step.sleep()?\"\n- \"How do I test that state changed from the seeded value?\"\n\n---\n\n## Response Format\n\n**Match depth to the question.**\n\n### Simple questions (\"what assertions are available?\", \"how do I run evals?\")\n\nAnswer directly — show the relevant table or CLI command. Don't generate a full eval file for an informational question.\n\n### Writing an eval\n\n1. Show the complete `new Eval({})` call with realistic field values\n2. Include imports (`import { Eval } from '@botpress\u002Fevals'`)\n3. Briefly explain non-obvious assertions — skip if the assertion is self-explanatory\n4. Suggest the CLI command to run it: `adk evals \u003Cname>`\n\n### Debugging a failing eval\n\n1. Ask for or show the failing assertion (`expected` \u002F `actual` diff)\n2. Suggest opening traces in the Dev Console to see what the bot did\n3. Identify whether the issue is in the eval assertion or the bot's behavior\n",{"name":5,"license":15,"description":7},"master","4aa004da461d820f4ea5499e58eb8d196cf99016","community\u002Fskills_sh\u002Fs\u002Fskills-sh_botpress_skills_adk-evals",0,"https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge","https:\u002F\u002Fskill-explorer-rho.vercel.app","https:\u002F\u002Fgithub.com\u002FLord1Egypt","Unbreakable. Unstoppable. Unmatched⚡.\r\n Speaker of machine tongues 🤖  | World-builder of decentralized dreams 🌐 | Defender of the digital realm🛡️.","2026-06-10 08:21:01",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add Lord1Egypt\u002Fawesome-skill-forge --skill adk-evals",{"tool":39,"command":40},"bunx","bunx skills add Lord1Egypt\u002Fawesome-skill-forge --skill adk-evals",{"tool":42,"command":43},"pnpm","pnpm dlx skills add Lord1Egypt\u002Fawesome-skill-forge --skill adk-evals","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fcommunity\u002Fskills_sh\u002Fs\u002Fskills-sh_botpress_skills_adk-evals\u002FSKILL.md",true,[],[48,55,62,69,76,83],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M785WKJB99G3STR1NF9NRTJW","youtube-content","YouTube transcripts to summaries, threads, blogs.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Fyoutube-content\u002FSKILL.md",[],[],{"skillNo":56,"name":57,"nameZh":6,"description":58,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":19},"01M785WEFHB0G2S9YA6EPR3H0E","xurl","X\u002FTwitter via xurl CLI: post, search, DM, media, v2 API.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Fxurl\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":65,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":68,"publishedAt":19},"01M785W98EDN4EFJXJPMQEHCRC","touchdesigner-mcp","Control a running TouchDesigner instance via twozero MCP — create operators, set parameters, wire connections, execute Python, build real-time visuals. 36 native tools.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Ftouchdesigner-mcp\u002FSKILL.md",[],[],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":19},"01M785W5S9FP3QBH2V191E025P","test-driven-development","TDD: enforce RED-GREEN-REFACTOR, tests before code.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Ftest-driven-development\u002FSKILL.md",[],[],{"skillNo":77,"name":78,"nameZh":6,"description":79,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":82,"publishedAt":19},"01M785W2AS08NSSH9YATCV6FJB","teams-meeting-pipeline","Operate the Teams meeting summary pipeline via Hermes CLI — summarize meetings, inspect pipeline status, replay jobs, manage Microsoft Graph subscriptions.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Fteams-meeting-pipeline\u002FSKILL.md",[],[],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":19},"01M785VXKW4T34BTP1SCHW1R7F","systematic-debugging","4-phase root cause debugging: understand bugs before fixing.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Fsystematic-debugging\u002FSKILL.md",[],[]]