[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78814B2XG95TZP40MFH02ED":3,"skill-same:01M78814B2XG95TZP40MFH02ED":69},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":25,"commitSha":26,"skillDirectory":27,"forksCount":14,"repositoryHtmlUrl":28,"repositoryHomepageUrl":6,"ownerHtmlUrl":29,"ownerBlogUrl":30,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M78814B2XG95TZP40MFH02ED","nemo-mbridge-perf-moe-vlm-training",null,"Practical guidance for training MoE VLMs in Megatron Bridge. Compares FSDP and 3D-parallel approaches, using rounded lessons from Qwen3-VL, Qwen3-Next, and other multimodal experiments.","Solizardking\u002Ftrain2earn","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786KDC5C3NPE4W44C5641EZ",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-vlm-training\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: nemo-mbridge-perf-moe-vlm-training\ndescription: Practical guidance for training MoE VLMs in Megatron Bridge. Compares FSDP and 3D-parallel approaches, using rounded lessons from Qwen3-VL, Qwen3-Next, and other multimodal experiments.\nlicense: Apache-2.0\nwhen_to_use: Training MoE VLMs, or investigating a commit that caused MoE VLM training failure or OOM; 'MoE VLM', 'multimodal MoE', 'Qwen3-VL training', 'FSDP vs 3D-parallel for VLM', 'MoE vision language model'.\n---\n\n# MoE VLM Training\n\nStable docs: @docs\u002Ftraining\u002Fmoe-optimization.md\nCard: @skills\u002Fnemo-mbridge-perf-moe-vlm-training\u002Fcard.yaml\n\n## FSDP vs 3D Parallel\n\n| Approach | Strength | Best fit |\n|---|---|---|\n| FSDP | Simplest path to a working multimodal run | first bring-up, memory-first tuning, awkward PP boundaries |\n| 3D parallel | Higher ceiling after tuning | stable models with a clean PP layout and time for deeper sweeps |\n\nFor MoE VLMs, the practical workflow is usually:\n\n1. get the first reliable run with FSDP\n2. stabilize real-data input, recompute, and memory behavior\n3. move to 3D parallel only if the throughput headroom is worth the extra work\n\n## Rounded Findings From Recent VLM Runs\n\n### Qwen3-VL class models\n\nThe main patterns were consistent across the tracker:\n\n- FSDP on GB200-class systems can already reach healthy high-teens utilization\n  with a comparatively simple setup\n- B200 FSDP runs are viable, but more sensitive to recompute choice and frozen\n  vision settings\n- 3D parallel can recover to a similar or better operating point, but only after\n  tuning MBS, recompute, and the real vision path together\n\n### Real data vs mock data\n\nMock-data VLM runs are not trustworthy performance proxies. In the experiments,\nimage-free mock runs looked closer to \"roughly twice as fast\" than \"slightly\noptimistic\" when compared with real multimodal input.\n\nUse real or realistic image payloads before drawing any conclusion about VLM\nthroughput.\n\n### Smaller multimodal MoE runs\n\nThe smaller Qwen3.5-style multimodal experiments reinforce the same lessons:\n\n- HybridEP is a solid default on GB200\n- TE-scoped CUDA graphs help once the training loop is stable\n- larger MBS can pay off, but only if the vision encoder does not become the\n  next bottleneck\n\n## Decision Guide\n\n### Choose FSDP when\n\n- you are bringing up a new VLM for the first time\n- the model has awkward stage boundaries across embedding, vision, and decoder\n- memory fit matters more than absolute throughput\n- you may freeze the vision stack during decoder-focused tuning\n\n### Choose 3D parallel when\n\n- the model is already stable under FSDP\n- the PP layout is clear and repeatable\n- you can sweep MBS, recompute, and CUDA-graph scope together\n- the goal is best steady-state throughput, not easiest bring-up\n\n## Key Tuning Knobs\n\n1. **Freeze the vision stack when appropriate**: if the work is decoder-focused,\n   freezing the vision side often gives a small but real throughput gain and\n   reduces memory pressure.\n\n2. **Sweep MBS aggressively**: VLMs are more MBS-sensitive than text-only MoE\n   runs because the vision path changes the compute-to-overhead balance.\n\n3. **Prefer selective recompute once the model fits**: full recompute is a\n   useful bring-up tool, but selective recompute is usually the better steady\n   state.\n\n4. **Match CUDA-graph scope to the workload**: `attn moe_router moe_preprocess`\n   is the safer MoE default, while narrower scopes can still be useful for\n   controlled experiments.\n\n5. **Use ETP only when EP alone is insufficient**: it can unlock a layout, but\n   it also introduces more communication and more tuning surface.\n\n## Representative Config Families\n\n### FSDP-first GB200 path\n\n```text\nTP=1  CP=1  PP=1\nEP sized to the expert topology, often large\nDispatcher: HybridEP on GB200-class systems\nRecompute: start with full, then relax toward selective recompute\n```\n\n### 3D-parallel GB200 path\n\n```text\nTP=1  CP=1  PP=1 or modest PP\nEP and ETP sized to the expert topology\nDispatcher: HybridEP\nCUDA Graph: start narrow, then widen only after the real-data path is stable\n```\n\n## Compatibility\n\n| Feature | FSDP | 3D parallel |\n|---|---|---|\n| HybridEP on GB200 | strong default | strong default once topology is stable |\n| CUDA graphs | useful after bring-up | useful, but more scope-sensitive |\n| Freeze vision | natural fit | possible, but less often used as the headline perf path |\n| Selective recompute | recommended | recommended |\n\n## Pitfalls\n\n1. **Mock multimodal data is misleading**: it can make the decoder look much\n   healthier than the real end-to-end VLM path.\n\n2. **The vision encoder can dominate unexpectedly**: profile encoder, projector,\n   and decoder separately before attributing everything to the dispatcher.\n\n3. **Do not compare FSDP and 3D-parallel runs with different effective work**:\n   normalize by useful tokens and workload shape, not only by step time.\n\n4. **ETP is not free**: use it as a fit or topology tool, not as the default.\n\n5. **Recompute and CUDA-graph choices are coupled**: the setting that gets the\n   model to fit is often not the setting that gives the best steady-state speed.\n","# MoE VLM Training\n\nStable docs: @docs\u002Ftraining\u002Fmoe-optimization.md\nCard: @skills\u002Fnemo-mbridge-perf-moe-vlm-training\u002Fcard.yaml\n\n## FSDP vs 3D Parallel\n\n| Approach | Strength | Best fit |\n|---|---|---|\n| FSDP | Simplest path to a working multimodal run | first bring-up, memory-first tuning, awkward PP boundaries |\n| 3D parallel | Higher ceiling after tuning | stable models with a clean PP layout and time for deeper sweeps |\n\nFor MoE VLMs, the practical workflow is usually:\n\n1. get the first reliable run with FSDP\n2. stabilize real-data input, recompute, and memory behavior\n3. move to 3D parallel only if the throughput headroom is worth the extra work\n\n## Rounded Findings From Recent VLM Runs\n\n### Qwen3-VL class models\n\nThe main patterns were consistent across the tracker:\n\n- FSDP on GB200-class systems can already reach healthy high-teens utilization\n  with a comparatively simple setup\n- B200 FSDP runs are viable, but more sensitive to recompute choice and frozen\n  vision settings\n- 3D parallel can recover to a similar or better operating point, but only after\n  tuning MBS, recompute, and the real vision path together\n\n### Real data vs mock data\n\nMock-data VLM runs are not trustworthy performance proxies. In the experiments,\nimage-free mock runs looked closer to \"roughly twice as fast\" than \"slightly\noptimistic\" when compared with real multimodal input.\n\nUse real or realistic image payloads before drawing any conclusion about VLM\nthroughput.\n\n### Smaller multimodal MoE runs\n\nThe smaller Qwen3.5-style multimodal experiments reinforce the same lessons:\n\n- HybridEP is a solid default on GB200\n- TE-scoped CUDA graphs help once the training loop is stable\n- larger MBS can pay off, but only if the vision encoder does not become the\n  next bottleneck\n\n## Decision Guide\n\n### Choose FSDP when\n\n- you are bringing up a new VLM for the first time\n- the model has awkward stage boundaries across embedding, vision, and decoder\n- memory fit matters more than absolute throughput\n- you may freeze the vision stack during decoder-focused tuning\n\n### Choose 3D parallel when\n\n- the model is already stable under FSDP\n- the PP layout is clear and repeatable\n- you can sweep MBS, recompute, and CUDA-graph scope together\n- the goal is best steady-state throughput, not easiest bring-up\n\n## Key Tuning Knobs\n\n1. **Freeze the vision stack when appropriate**: if the work is decoder-focused,\n   freezing the vision side often gives a small but real throughput gain and\n   reduces memory pressure.\n\n2. **Sweep MBS aggressively**: VLMs are more MBS-sensitive than text-only MoE\n   runs because the vision path changes the compute-to-overhead balance.\n\n3. **Prefer selective recompute once the model fits**: full recompute is a\n   useful bring-up tool, but selective recompute is usually the better steady\n   state.\n\n4. **Match CUDA-graph scope to the workload**: `attn moe_router moe_preprocess`\n   is the safer MoE default, while narrower scopes can still be useful for\n   controlled experiments.\n\n5. **Use ETP only when EP alone is insufficient**: it can unlock a layout, but\n   it also introduces more communication and more tuning surface.\n\n## Representative Config Families\n\n### FSDP-first GB200 path\n\n```text\nTP=1  CP=1  PP=1\nEP sized to the expert topology, often large\nDispatcher: HybridEP on GB200-class systems\nRecompute: start with full, then relax toward selective recompute\n```\n\n### 3D-parallel GB200 path\n\n```text\nTP=1  CP=1  PP=1 or modest PP\nEP and ETP sized to the expert topology\nDispatcher: HybridEP\nCUDA Graph: start narrow, then widen only after the real-data path is stable\n```\n\n## Compatibility\n\n| Feature | FSDP | 3D parallel |\n|---|---|---|\n| HybridEP on GB200 | strong default | strong default once topology is stable |\n| CUDA graphs | useful after bring-up | useful, but more scope-sensitive |\n| Freeze vision | natural fit | possible, but less often used as the headline perf path |\n| Selective recompute | recommended | recommended |\n\n## Pitfalls\n\n1. **Mock multimodal data is misleading**: it can make the decoder look much\n   healthier than the real end-to-end VLM path.\n\n2. **The vision encoder can dominate unexpectedly**: profile encoder, projector,\n   and decoder separately before attributing everything to the dispatcher.\n\n3. **Do not compare FSDP and 3D-parallel runs with different effective work**:\n   normalize by useful tokens and workload shape, not only by step time.\n\n4. **ETP is not free**: use it as a fit or topology tool, not as the default.\n\n5. **Recompute and CUDA-graph choices are coupled**: the setting that gets the\n   model to fit is often not the setting that gives the best steady-state speed.\n",{"name":5,"license":15,"description":7,"when_to_use":24},"Training MoE VLMs, or investigating a commit that caused MoE VLM training failure or OOM; 'MoE VLM', 'multimodal MoE', 'Qwen3-VL training', 'FSDP vs 3D-parallel for VLM', 'MoE vision language model'.","main","86cc27ffd16af6cb85948f1155578e6f3f61cc4c","skillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-vlm-training","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 11:52:53",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add Solizardking\u002Ftrain2earn --skill nemo-mbridge-perf-moe-vlm-training",{"tool":39,"command":40},"bunx","bunx skills add Solizardking\u002Ftrain2earn --skill nemo-mbridge-perf-moe-vlm-training",{"tool":42,"command":43},"pnpm","pnpm dlx skills add Solizardking\u002Ftrain2earn --skill nemo-mbridge-perf-moe-vlm-training","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-vlm-training\u002FSKILL.md",true,[47,53,57,61,65],{"path":48,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":51,"sourceHtmlUrl":52},"BENCHMARK.md","REFERENCE","参考文档",4355,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-vlm-training\u002FBENCHMARK.md",{"path":54,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":55,"sourceHtmlUrl":56},"card.yaml",2993,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-vlm-training\u002Fcard.yaml",{"path":58,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":59,"sourceHtmlUrl":60},"metadata.json",437,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-vlm-training\u002Fmetadata.json",{"path":62,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":63,"sourceHtmlUrl":64},"skill-card.md",3931,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-vlm-training\u002Fskill-card.md",{"path":66,"resourceType":49,"resourceTypeDesc":50,"sizeBytes":67,"sourceHtmlUrl":68},"verification.json",1590,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-vlm-training\u002Fverification.json",[70,78,86,94,101,109],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":74,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":19},"01M78B94RSCAKMH1QS79VE6DG9","testing","For testing with Light Protocol programs and clients on localnet, devnet, and mainnet validation.","MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Ftesting\u002FSKILL.md",[],[],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":74,"licenseFileUrl":82,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":19},"01M78B81BKY8RX12HETPG8YKWK","qedgen","Formally verify programs by writing Lean 4 proofs. Trigger this skill whenever the user wants to formally verify code, generate Lean 4 proofs, prove properties about algorithms or smart contracts, verify invariants, convert program logic into formal specifications, or anything involving Lean 4 and formal verification. Also trigger when the user mentions \"qedgen\", \"lean proof\", \"formal proof\", \"verify my code\", \"prove correctness\", \"formal verification\", or wants mathematical guarantees about their implementation.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-formal-verification\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-formal-verification\u002FSKILL.md",[],[],{"skillNo":87,"name":88,"nameZh":6,"description":89,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":74,"licenseFileUrl":90,"sourceHtmlUrl":91,"categories":92,"keywords":93,"publishedAt":19},"01M78B7Q7C5K54E01DN83A60KG","solana-dev","Use when user asks to \"build a Solana dapp\", \"write an Anchor program\", \"create a token\", \"debug Solana errors\", \"set up wallet connection\", \"test my Solana program\", \"deploy to devnet\", or \"explain Solana concepts\" (rent, accounts, PDAs, CPIs, etc.). Also use for quick on-chain lookups via public RPC + curl — \"what's the balance of \u003Cwallet>\", \"look up transaction \u003Csig>\", \"token balance for \u003Caccount>\", \"check this address on mainnet\u002Fdevnet\". End-to-end Solana development playbook covering wallet connection, Anchor\u002FPinocchio programs, Codama client generation, LiteSVM\u002FMollusk\u002FSurfpool testing, security checklists, and JSON-RPC curl lookups against public clusters. Integrates with the Solana MCP server for live documentation search. Prefers framework-kit (@solana\u002Fclient + @solana\u002Freact-hooks) for UI, wallet-standard-first connection (incl. ConnectorKit), @solana\u002Fkit for client\u002FRPC code, and @solana\u002Fweb3-compat for legacy boundaries.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-dev\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-dev\u002FSKILL.md",[],[],{"skillNo":95,"name":96,"nameZh":6,"description":97,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":74,"licenseFileUrl":6,"sourceHtmlUrl":98,"categories":99,"keywords":100,"publishedAt":19},"01M78B7HXD6N6GWKSVRQ8F6Q31","solana-common-errors","Diagnose and fix common Solana development errors — GLIBC mismatches, Anchor version conflicts, cargo build-sbf \u002F platform-tools failures, LiteSVM issues, edition2024 crate pins, RPC airdrops, and Anchor 0.29→0.32 migrations. Use when builds fail, Anchor CLI won't install, or CI breaks on Debian\u002FUbuntu.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-common-errors\u002FSKILL.md",[],[],{"skillNo":102,"name":103,"nameZh":6,"description":104,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":105,"sourceHtmlUrl":106,"categories":107,"keywords":108,"publishedAt":19},"01M78B6T7YSYWQ0N4DP40QDXEZ","skill-creator","Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fskill-creator\u002Flicense.txt","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fskill-creator\u002FSKILL.md",[],[],{"skillNo":110,"name":111,"nameZh":6,"description":112,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":74,"licenseFileUrl":6,"sourceHtmlUrl":113,"categories":114,"keywords":115,"publishedAt":19},"01M78B1VFT9X5AND7AE1HZPZGF","phantom-wallet-mcp","Execute wallet operations through the Phantom MCP server. Use when the user wants to interact with their Phantom wallet directly — get addresses, transfer SOL or SPL tokens, buy\u002Fswap tokens, sign transactions, and sign messages across Solana, Ethereum, Bitcoin, and Sui. Requires the @phantom\u002Fmcp-server to be configured as an MCP server.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fphantom-wallet-mcp\u002FSKILL.md",[],[]]