[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MCDFS8FBFSD8SJTM50G7GG":3,"skill-same:01M6MCDFS8FBFSD8SJTM50G7GG":113},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":34,"publishedAt":56,"linkOnly":57,"markdownContent":58,"markdownBody":59,"frontMatter":60,"refName":67,"commitSha":68,"skillDirectory":69,"forksCount":70,"repositoryHtmlUrl":71,"repositoryHomepageUrl":72,"ownerHtmlUrl":73,"ownerBlogUrl":74,"ownerDescription":75,"updatedAt":76,"install":77,"resources":90},"01M6MCDFS8FBFSD8SJTM50G7GG","holdback-experiment-design",null,"Design degradation holdbacks and long-term cumulative holdbacks for product experiments and feature rollouts. Use when a team needs a long-term counterfactual, wants to measure delayed impact after launch, is worried about metric degradation over time, needs to decide holdback size or duration, or must weigh the user\u002Fbusiness cost of withholding a feature.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1559,"MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FLVTD-LLC\u002Fskills\u002Fskills\u002Fholdback-experiment-design\u002FSKILL.md",[18,23,26,31],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000015","视觉设计","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":27,"categoryName":28,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000031","产品经理","PROFESSION","适用职业",{"categoryNo":32,"categoryName":33,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000032","设计师",[35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55],"design","holdback experiment design","experiment design","experiment","when","testing","metric","ab-testing","experimentation","impact","ab","feature","time","practical","rollouts","decide","team","product","code","analytics","experiments","2026-11-10 10:12:30",false,"---\nname: holdback-experiment-design\ndescription: Design degradation holdbacks and long-term cumulative holdbacks for product experiments and feature rollouts. Use when a team needs a long-term counterfactual, wants to measure delayed impact after launch, is worried about metric degradation over time, needs to decide holdback size or duration, or must weigh the user\u002Fbusiness cost of withholding a feature.\nlicense: MIT\ncompatibility: Codex, Claude Code, and other Agent Skills-compatible clients.\nmetadata:\n  version: \"0.1.0\"\n  displayName: Holdback Experiment Design\n  category: Product Management\n  tags: practical-ab-testing,ab-testing,experimentation,holdbacks,product-analytics\n---\n\n# Holdback Experiment Design\n\nUse this skill to plan holdbacks that preserve a comparison group after a\nfeature rollout. Holdbacks are useful when effects may appear later, accumulate,\nor degrade after launch.\n\n## Source Traceability\n\nPrimary source: Practical A\u002FB Testing by Leemay Nassery. Guidance is\ntransformed and paraphrased from chapter 3 lines 2487-2836. Experiment-type\ncontext comes from chapter 3 lines 2013-2486.\n\n## Related Advanced Skills\n\n- `long-term-impact-evaluation`: use before detailed holdback design when the\n  team should compare holdbacks against post-period analysis, continuous\n  monitoring, CLV models, or hybrid methods.\n- `trustworthy-experiment-insights`: use when deciding whether long-term or\n  holdback evidence is credible enough to drive a product decision.\n\n## Reference Routing\n\n| Need | Read |\n|------|------|\n| Holdback concepts | `references\u002Fcore\u002Fknowledge.md` |\n| Design and cost rules | `references\u002Fcore\u002Frules.md` |\n| Scenario examples | `references\u002Fcore\u002Fexamples.md` |\n| Step-by-step planning | `workflows\u002Fdesign-holdback.md` |\n\n## Workflow\n\n1. State why short-term A\u002FB evidence is insufficient.\n2. Choose degradation or long-term cumulative holdback.\n3. Define who remains withheld, for how long, and from what experience.\n4. Select long-term metrics and guardrails.\n5. Estimate the user, business, and ethical cost of withholding.\n6. Define monitoring cadence, exit criteria, and communication plan.\n\n## Output Format\n\n```markdown\n# Holdback Plan\n\n## Purpose\n[What long-term question this holdback answers.]\n\n## Holdback Type\n[Degradation | Long-term cumulative]\n\n## Population And Duration\n- Holdback population:\n- Rollout population:\n- Duration:\n- Removal criteria:\n\n## Metrics\n| Metric | Role | Readout Cadence | Concern |\n|--------|------|-----------------|---------|\n\n## Cost Of Withholding\n- User cost:\n- Business cost:\n- Ethical or trust concern:\n\n## Decision Rules\n- Continue holdback if:\n- End holdback if:\n- Escalate if:\n```\n\n## Quality Bar\n\n- Do not create a holdback without a specific long-term question.\n- Do not withhold a clearly valuable feature longer than the question requires.\n- Do not ignore the opportunity cost to held-back users.\n- Monitor guardrails while the holdback is active.\n","# Holdback Experiment Design\n\nUse this skill to plan holdbacks that preserve a comparison group after a\nfeature rollout. Holdbacks are useful when effects may appear later, accumulate,\nor degrade after launch.\n\n## Source Traceability\n\nPrimary source: Practical A\u002FB Testing by Leemay Nassery. Guidance is\ntransformed and paraphrased from chapter 3 lines 2487-2836. Experiment-type\ncontext comes from chapter 3 lines 2013-2486.\n\n## Related Advanced Skills\n\n- `long-term-impact-evaluation`: use before detailed holdback design when the\n  team should compare holdbacks against post-period analysis, continuous\n  monitoring, CLV models, or hybrid methods.\n- `trustworthy-experiment-insights`: use when deciding whether long-term or\n  holdback evidence is credible enough to drive a product decision.\n\n## Reference Routing\n\n| Need | Read |\n|------|------|\n| Holdback concepts | `references\u002Fcore\u002Fknowledge.md` |\n| Design and cost rules | `references\u002Fcore\u002Frules.md` |\n| Scenario examples | `references\u002Fcore\u002Fexamples.md` |\n| Step-by-step planning | `workflows\u002Fdesign-holdback.md` |\n\n## Workflow\n\n1. State why short-term A\u002FB evidence is insufficient.\n2. Choose degradation or long-term cumulative holdback.\n3. Define who remains withheld, for how long, and from what experience.\n4. Select long-term metrics and guardrails.\n5. Estimate the user, business, and ethical cost of withholding.\n6. Define monitoring cadence, exit criteria, and communication plan.\n\n## Output Format\n\n```markdown\n# Holdback Plan\n\n## Purpose\n[What long-term question this holdback answers.]\n\n## Holdback Type\n[Degradation | Long-term cumulative]\n\n## Population And Duration\n- Holdback population:\n- Rollout population:\n- Duration:\n- Removal criteria:\n\n## Metrics\n| Metric | Role | Readout Cadence | Concern |\n|--------|------|-----------------|---------|\n\n## Cost Of Withholding\n- User cost:\n- Business cost:\n- Ethical or trust concern:\n\n## Decision Rules\n- Continue holdback if:\n- End holdback if:\n- Escalate if:\n```\n\n## Quality Bar\n\n- Do not create a holdback without a specific long-term question.\n- Do not withhold a clearly valuable feature longer than the question requires.\n- Do not ignore the opportunity cost to held-back users.\n- Monitor guardrails while the holdback is active.\n",{"name":5,"license":15,"metadata":61,"description":7,"compatibility":66},{"tags":62,"version":63,"category":64,"displayName":65},"practical-ab-testing,ab-testing,experimentation,holdbacks,product-analytics","0.1.0","Product Management","Holdback Experiment Design","Codex, Claude Code, and other Agent Skills-compatible clients.","main","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002FLVTD-LLC\u002Fskills\u002Fskills\u002Fholdback-experiment-design",474,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-11 14:40:14",{"source":8,"skillName":5,"commands":78,"prompt":88,"zipAvailable":89},[79,82,85],{"tool":80,"command":81},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill holdback-experiment-design",{"tool":83,"command":84},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill holdback-experiment-design",{"tool":86,"command":87},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill holdback-experiment-design","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FLVTD-LLC\u002Fskills\u002Fskills\u002Fholdback-experiment-design\u002FSKILL.md",true,[91,97,101,105,109],{"path":92,"resourceType":93,"resourceTypeDesc":94,"sizeBytes":95,"sourceHtmlUrl":96},"guidelines.md","REFERENCE","参考文档",1363,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FLVTD-LLC\u002Fskills\u002Fskills\u002Fholdback-experiment-design\u002Fguidelines.md",{"path":98,"resourceType":93,"resourceTypeDesc":94,"sizeBytes":99,"sourceHtmlUrl":100},"references\u002Fcore\u002Fexamples.md",1467,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FLVTD-LLC\u002Fskills\u002Fskills\u002Fholdback-experiment-design\u002Freferences\u002Fcore\u002Fexamples.md",{"path":102,"resourceType":93,"resourceTypeDesc":94,"sizeBytes":103,"sourceHtmlUrl":104},"references\u002Fcore\u002Fknowledge.md",1803,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FLVTD-LLC\u002Fskills\u002Fskills\u002Fholdback-experiment-design\u002Freferences\u002Fcore\u002Fknowledge.md",{"path":106,"resourceType":93,"resourceTypeDesc":94,"sizeBytes":107,"sourceHtmlUrl":108},"references\u002Fcore\u002Frules.md",1632,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FLVTD-LLC\u002Fskills\u002Fskills\u002Fholdback-experiment-design\u002Freferences\u002Fcore\u002Frules.md",{"path":110,"resourceType":93,"resourceTypeDesc":94,"sizeBytes":111,"sourceHtmlUrl":112},"workflows\u002Fdesign-holdback.md",1877,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FLVTD-LLC\u002Fskills\u002Fskills\u002Fholdback-experiment-design\u002Fworkflows\u002Fdesign-holdback.md",[114,123,130,137,161,186],{"skillNo":115,"name":116,"nameZh":6,"description":117,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":118,"licenseFileUrl":6,"sourceHtmlUrl":119,"categories":120,"keywords":121,"publishedAt":122},"01M746ZWACVKMZ1BJXJZHYJSSX","migrate-to-e2e","Moves an existing Playwright, Cypress, Selenium, Appium, Detox, Maestro, or TesterArmy suite to e2e one slice at a time, with the old suite running beside the new one until each ported test is proven. Keeps a MIGRATION.md progress file so any later session resumes where the last stopped. Optionally plants a defect to show both suites catch it. Use for \u002Fmigrate-to-e2e, \"migrate our tests to e2e\", \"port these Maestro flows\", \"move our TesterArmy tests\", or when a project has an e2e config and an older suite still running.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fmigrate-to-e2e\u002FSKILL.md",[],[],"2026-11-11 15:11:14",{"skillNo":124,"name":125,"nameZh":6,"description":126,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":118,"licenseFileUrl":6,"sourceHtmlUrl":127,"categories":128,"keywords":129,"publishedAt":122},"01M746ZPTAX1Q0FBKAB1CYND7E","e2e","Writes, runs, and debugs end-to-end tests for web and mobile apps with e2e, the agentic test runner, which mixes agent.act goals with exact locator assertions on a Playwright browser or an iOS or Android device. Also drives a project's running app live over the e2e MCP server to look at a screen, check a UI change, take a screenshot or record a video, and try a locator; and runs bug bashes, parallel e2e explore runs with each bug proven by a repro test. Use when a project depends on e2e or has an e2e.config.ts, when asked for end-to-end, browser, mobile, or agentic UI tests, to check a change in the running app, to QA or bug bash an app or branch, or when an e2e run fails.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fe2e\u002FSKILL.md",[],[],{"skillNo":131,"name":132,"nameZh":6,"description":133,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":118,"licenseFileUrl":6,"sourceHtmlUrl":134,"categories":135,"keywords":136,"publishedAt":122},"01M746ZJN784HVHMDP9MQGCSGG","create-verification-skill","Generate a project-local verify-\u003Capp> skill on top of e2e, so any coding agent can launch the app, check it is worth driving, exercise a feature the way a user does, keep evidence, and bug bash it. Writes the skill, a feature map with one explore charter per feature, and proves it once. Use for \u002Fcreate-verification-skill, \"make a verification skill for this repo\", or when a project has no scripted way to prove UI behavior.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fcreate-verification-skill\u002FSKILL.md",[],[],{"skillNo":138,"name":139,"nameZh":6,"description":140,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":118,"licenseFileUrl":6,"sourceHtmlUrl":141,"categories":142,"keywords":143,"publishedAt":56},"01M716A6ETNB9B1SQ3GXZC75VR","deja-search","deja-vu memory — search the user's past AI coding sessions with the deja CLI. Holds what they already settled and never wrote into the repo (naming rules, banned or already-tried tools, required checks before merge, preferred values, who owns what). Use before your first edit in a task, before you call a change done or ready to merge, before you recommend a tool, library, design or reviewer, before re-debugging an error that may already be solved, when they say 'didn't we fix this before', 'what did we decide about X' or 'I already have that', and before telling them something on this machine does not exist.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-search\u002FSKILL.md",[],[144,145,146,147,148,149,150,50,151,35,152,39,153,154,155,156,157,158,159,160],"search","deja search","never","error","ready","memory","re","fix","debugging","you","done","reviewer","io","change","cli","merge","past",{"skillNo":162,"name":163,"nameZh":6,"description":164,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":118,"licenseFileUrl":6,"sourceHtmlUrl":165,"categories":166,"keywords":167,"publishedAt":56},"01M716A3J2W5SENSE9VGMXB375","deja-reflect","End-of-session reflection and memory curation with deja-vu. Review what was decided, what commands broke and what repaired them, and record durable project rules or decisions with deja remember so the next session starts with them.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-reflect\u002FSKILL.md",[],[168,169,170,156,171,172,173,174,175,176,177,178,179,180,149,181,182,183,184,185],"deja reflect","reflect","review","with","record","github","next","https","node","Rules","decisions","brew","remember","project rules","project","session","end","reflection",{"skillNo":187,"name":188,"nameZh":6,"description":189,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":118,"licenseFileUrl":6,"sourceHtmlUrl":190,"categories":191,"keywords":199,"publishedAt":56},"01M715T3RR4BAYSR57HPPTJ0PF","revid","Make, edit, export and post AI videos with Revid (revid.ai) through the Revid MCP server. Use whenever the user wants a short-form or long-form video made (from an idea, a script, a URL, an article, a PDF, a tweet, a product page, their own clips, or a song), wants to find what is going viral on TikTok, YouTube, Instagram or X before making something, wants to edit or re-export an existing Revid project, generate a standalone AI image or clip, make a series episode, or publish and schedule a finished video to connected social accounts. Covers setup (hosted MCP with OAuth, or an API key), the 36 tools, credits, polling and the rules for publishing. Routes to the more specific revid-* skills when one fits.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftmakersas\u002Frevid-skills\u002Fskills\u002Frevid\u002FSKILL.md",[192,195,196],{"categoryNo":193,"categoryName":194,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":197,"categoryName":198,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口",[188,200,201,202,203,204,205,182,206,150,207,208,209,210,39,211,212,213,214,215],"URL","Video","short-form","standalone","PDF","viral","tiktok","short","social","MCP","song","image","mcp server","schedule","api","publish"]