[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6FJVE1G6TP6CEX6J4JJH5SW":3,"skill-same:01M6FJVE1G6TP6CEX6J4JJH5SW":48},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":21,"linkOnly":22,"markdownContent":23,"markdownBody":23,"frontMatter":24,"refName":25,"commitSha":26,"skillDirectory":27,"forksCount":28,"repositoryHtmlUrl":29,"repositoryHomepageUrl":30,"ownerHtmlUrl":31,"ownerBlogUrl":6,"ownerDescription":32,"updatedAt":33,"install":34,"resources":47},"01M6FJVE1G6TP6CEX6J4JJH5SW","experiment-plan",null,"ComeOnOliver\u002Fskillshub","ComeOnOliver","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F43215937?v=4","01M6E5JNFECJAAVFE0W7SMWWD2","Haoran Shu","01M6E5JNNDQS7AJZDYFQ8TPZCK",67,"MIT","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fwanshuiyin\u002FAuto-claude-code-research-in-sleep\u002Fexperiment-plan\u002FSKILL.md",[],[18,19,20],"plan","experiment","experiment plan","2026-11-10 10:12:30",false,"# Experiment Plan: Claim-Driven, Paper-Oriented Validation\n\nRefine and concretize: **$ARGUMENTS**\n\n## Overview\n\nUse this skill after the method is stable enough that the next question becomes: **what exact experiments should we run, in what order, to defend the paper?** If the user wants the full chain in one request, prefer `\u002Fresearch-refine-pipeline`.\n\nThe goal is not to generate a giant benchmark wishlist. The goal is to turn a proposal into a **claim -> evidence -> run order** roadmap that supports four things:\n\n1. the method actually solves the anchored problem\n2. the dominant contribution is real and focused\n3. the method is elegant enough that extra complexity is unnecessary\n4. any frontier-model-era component is genuinely useful, not decorative\n\n## Constants\n\n- **OUTPUT_DIR = `refine-logs\u002F`** — Default destination for experiment planning artifacts.\n- **MAX_PRIMARY_CLAIMS = 2** — Prefer one dominant claim plus one supporting claim.\n- **MAX_CORE_BLOCKS = 5** — Keep the must-run experimental story compact.\n- **MAX_BASELINE_FAMILIES = 3** — Prefer a few strong baselines over many weak ones.\n- **DEFAULT_SEEDS = 3** — Use 3 seeds when stochastic variance matters and budget allows.\n\n## Workflow\n\n### Phase 0: Load the Proposal Context\n\nRead the most relevant existing files first if they exist:\n\n- `refine-logs\u002FFINAL_PROPOSAL.md`\n- `refine-logs\u002FREVIEW_SUMMARY.md`\n- `refine-logs\u002FREFINEMENT_REPORT.md`\n\nExtract:\n\n- **Problem Anchor**\n- **Dominant contribution**\n- **Optional supporting contribution**\n- **Critical reviewer concerns**\n- **Data \u002F compute \u002F timeline constraints**\n- **Which frontier primitive is central, if any**\n\nIf these files do not exist, derive the same information from the user's prompt.\n\n### Phase 1: Freeze the Paper Claims\n\nBefore proposing experiments, write down the claims that must be defended.\n\nUse this structure:\n\n- **Primary claim**: the main mechanism-level contribution\n- **Supporting claim**: optional, only if it directly strengthens the main paper story\n- **Anti-claim to rule out**: e.g. \"the gain only comes from more parameters,\" \"the gain only comes from a larger search space,\" or \"the modern component is just decoration\"\n- **Minimum convincing evidence**: what would make each claim believable to a strong reviewer?\n\nDo not exceed `MAX_PRIMARY_CLAIMS` unless the paper truly has multiple inseparable claims.\n\n### Phase 2: Build the Experimental Storyline\n\nDesign the paper around a compact set of experiment blocks. Default to the following blocks and delete any that are not needed:\n\n1. **Main anchor result** — does the method solve the actual bottleneck?\n2. **Novelty isolation** — does the dominant contribution itself matter?\n3. **Simplicity \u002F elegance check** — can a bigger or more fragmented version be avoided?\n4. **Frontier necessity check** — if an LLM \u002F VLM \u002F Diffusion \u002F RL-era component is central, is it actually the right tool?\n5. **Failure analysis or qualitative diagnosis** — what does the method still miss?\n\nFor each block, decide whether it belongs in:\n\n- **Main paper** — essential to defend the core claims\n- **Appendix** — useful but non-blocking\n- **Cut** — interesting, but not worth the paper budget\n\nPrefer one strong baseline family over many weak baselines. If a stronger modern baseline exists, use it instead of padding the list.\n\n### Phase 3: Specify Each Experiment Block\n\nFor every kept block, fully specify:\n\n- **Claim tested**\n- **Why this block exists**\n- **Dataset \u002F split \u002F task**\n- **Compared systems**: strongest baselines, ablations, and variants only\n- **Metrics**: decisive metrics first, secondary metrics second\n- **Setup details**: backbone, frozen vs trainable parts, key hyperparameters, training budget, seeds\n- **Success criterion**: what outcome would count as convincing evidence?\n- **Failure interpretation**: if the result is negative, what does it mean?\n- **Table \u002F figure target**: where this result should appear in the paper\n\nSpecial rules:\n\n- A **simplicity check** should usually compare the final method against either an overbuilt variant or a tempting extra component that the paper intentionally rejects.\n- A **frontier necessity check** should usually compare the chosen modern primitive against the strongest plausible simpler or older alternative.\n- If the proposal is intentionally non-frontier, say so explicitly and skip the frontier block instead of forcing one.\n\n### Phase 4: Turn the Plan Into an Execution Order\n\nBuild a realistic run order so the user knows what to do first.\n\nUse this milestone structure:\n\n1. **Sanity stage** — data pipeline, metric correctness, one quick overfit or toy split\n2. **Baseline stage** — reproduce the strongest baseline(s)\n3. **Main method stage** — run the final method on the primary setting\n4. **Decision stage** — run the decisive ablations for novelty, simplicity, and frontier necessity\n5. **Polish stage** — robustness, qualitative figures, appendix extras\n\nFor each milestone, estimate:\n\n- compute cost\n- expected turnaround time\n- stop \u002F go decision gate\n- risk and mitigation\n\nSeparate **must-run** from **nice-to-have** experiments.\n\n### Phase 5: Write the Outputs\n\n#### Step 5.1: Write `refine-logs\u002FEXPERIMENT_PLAN.md`\n\nUse this structure:\n\n```markdown\n# Experiment Plan\n\n**Problem**: [problem]\n**Method Thesis**: [one-sentence thesis]\n**Date**: [today]\n\n## Claim Map\n| Claim | Why It Matters | Minimum Convincing Evidence | Linked Blocks |\n|-------|-----------------|-----------------------------|---------------|\n| C1    | ...             | ...                         | B1, B2        |\n\n## Paper Storyline\n- Main paper must prove:\n- Appendix can support:\n- Experiments intentionally cut:\n\n## Experiment Blocks\n\n### Block 1: [Name]\n- Claim tested:\n- Why this block exists:\n- Dataset \u002F split \u002F task:\n- Compared systems:\n- Metrics:\n- Setup details:\n- Success criterion:\n- Failure interpretation:\n- Table \u002F figure target:\n- Priority: MUST-RUN \u002F NICE-TO-HAVE\n\n### Block 2: [Name]\n...\n\n## Run Order and Milestones\n| Milestone | Goal | Runs | Decision Gate | Cost | Risk |\n|-----------|------|------|---------------|------|------|\n| M0        | ...  | ...  | ...           | ...  | ...  |\n\n## Compute and Data Budget\n- Total estimated GPU-hours:\n- Data preparation needs:\n- Human evaluation needs:\n- Biggest bottleneck:\n\n## Risks and Mitigations\n- [Risk]:\n- [Mitigation]:\n\n## Final Checklist\n- [ ] Main paper tables are covered\n- [ ] Novelty is isolated\n- [ ] Simplicity is defended\n- [ ] Frontier contribution is justified or explicitly not claimed\n- [ ] Nice-to-have runs are separated from must-run runs\n```\n\n#### Step 5.2: Write `refine-logs\u002FEXPERIMENT_TRACKER.md`\n\nUse this structure:\n\n```markdown\n# Experiment Tracker\n\n| Run ID | Milestone | Purpose | System \u002F Variant | Split | Metrics | Priority | Status | Notes |\n|--------|-----------|---------|------------------|-------|---------|----------|--------|-------|\n| R001   | M0        | sanity  | ...              | ...   | ...     | MUST     | TODO   | ...   |\n```\n\nKeep the tracker compact and execution-oriented.\n\n#### Step 5.3: Present a Brief Summary to the User\n\n```\nExperiment plan ready.\n\nMust-run blocks:\n- [Block 1]\n- [Block 2]\n\nHighest-risk assumption:\n- [risk]\n\nFirst three runs to launch:\n1. [run]\n2. [run]\n3. [run]\n\nPlan file: refine-logs\u002FEXPERIMENT_PLAN.md\nTracker file: refine-logs\u002FEXPERIMENT_TRACKER.md\n```\n\n## Key Rules\n\n- **Large file handling**: If the Write tool fails due to file size, immediately retry using Bash (`cat \u003C\u003C 'EOF' > file`) to write in chunks. Do NOT ask the user for permission — just do it silently.\n\n- **Every experiment must defend a claim.** If it does not change a reviewer belief, cut it.\n- **Prefer a compact paper story.** Design the main table first, then add only the ablations that defend it.\n- **Defend simplicity explicitly.** If complexity is a concern, include a deletion study or a stronger-but-bloated variant comparison.\n- **Defend frontier choices explicitly.** If a modern primitive is central, prove why it is better than the strongest simpler alternative.\n- **Prefer strong baselines over long baseline lists.** A short, credible comparison set is better than a padded one.\n- **Separate must-run from nice-to-have.** Do not let appendix ideas delay the core paper evidence.\n- **Reuse proposal constraints.** Do not invent unrealistic budgets or data assumptions.\n- **Do not fabricate results.** Plan evidence; do not claim evidence.\n\n## Composing with Other Skills\n\n```\n\u002Fresearch-refine-pipeline -> one-shot method + experiment planning\n\u002Fresearch-refine   -> method and claim refinement\n\u002Fexperiment-plan   -> detailed experiment roadmap\n\u002Frun-experiment    -> execute the runs\n\u002Fauto-review-loop  -> react to results and iterate on the paper\n```\n",{},"main","def8531e65114c0fca8fb8551c1871ee0eed705c","skills\u002Fwanshuiyin\u002FAuto-claude-code-research-in-sleep\u002Fexperiment-plan",24,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub","https:\u002F\u002Fskillshub.wtf","https:\u002F\u002Fgithub.com\u002FComeOnOliver","AI & Blockchain Engineer\r\n","2026-06-24 22:07:19",{"source":7,"skillName":5,"commands":35,"prompt":45,"zipAvailable":46},[36,39,42],{"tool":37,"command":38},"npx","npx skills add ComeOnOliver\u002Fskillshub --skill experiment-plan",{"tool":40,"command":41},"bunx","bunx skills add ComeOnOliver\u002Fskillshub --skill experiment-plan",{"tool":43,"command":44},"pnpm","pnpm dlx skills add ComeOnOliver\u002Fskillshub --skill experiment-plan","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fwanshuiyin\u002FAuto-claude-code-research-in-sleep\u002Fexperiment-plan\u002FSKILL.md",true,[],[49,63,73,82,88,95],{"skillNo":50,"name":51,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":59,"publishedAt":21},"01M6FKJN13GMG4KY57J3QD8GY5","model-hierarchy-skill","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzscole\u002Fmodel-hierarchy-skill\u002Fmodel-hierarchy-skill\u002FSKILL.md",[54],{"categoryNo":55,"categoryName":56,"taxonomyType":57,"taxonomyTypeDesc":58},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[60,61,62],"model","model hierarchy","model hierarchy skill",{"skillNo":64,"name":65,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":69,"publishedAt":21},"01M6FKJJJ6GCAMR4PPPRD12M5X","zoom-mcp","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fzoom-mcp\u002FSKILL.md",[68],{"categoryNo":55,"categoryName":56,"taxonomyType":57,"taxonomyTypeDesc":58},[70,71,72],"MCP","zoom","zoom mcp",{"skillNo":74,"name":75,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":21},"01M6FKJE3KYT3KPEZMEEHYKA6K","zoom-apps-sdk","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fzoom-apps-sdk\u002FSKILL.md",[],[79,80,81,71],"sdk","zoom apps sdk","apps",{"skillNo":83,"name":84,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":87,"publishedAt":21},"01M6FKJ9EDH9Q5VWDF8HAJRSP0","windows","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwindows\u002FSKILL.md",[],[84],{"skillNo":89,"name":90,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":91,"categories":92,"keywords":94,"publishedAt":21},"01M6FKJ666BEEFC3AE9GJ72Q93","whiteboard","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwhiteboard\u002FSKILL.md",[93],{"categoryNo":55,"categoryName":56,"taxonomyType":57,"taxonomyTypeDesc":58},[90],{"skillNo":96,"name":97,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":98,"categories":99,"keywords":100,"publishedAt":21},"01M6FKJ21WFCDGTJQ2Q91T9DVW","websockets","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwebsockets\u002FSKILL.md",[],[97]]