[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78D09YXW9MTCDPWFZNW3B7P":3,"skill-same:01M78D09YXW9MTCDPWFZNW3B7P":72},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":25,"commitSha":26,"skillDirectory":27,"forksCount":14,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":31,"ownerDescription":32,"updatedAt":33,"install":34,"resources":47},"01M78D09YXW9MTCDPWFZNW3B7P","nemo-mbridge-perf-moe-comm-overlap",null,"MoE expert-parallel communication overlap in Megatron Bridge. Covers dispatch\u002Fcombine overlap, flex dispatcher backends, and expert wgrad scheduling.","Solizardking\u002Fskillhub-main","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786NYYXSZK1D05WM8JQPC48",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-comm-overlap\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: nemo-mbridge-perf-moe-comm-overlap\ndescription: MoE expert-parallel communication overlap in Megatron Bridge. Covers dispatch\u002Fcombine overlap, flex dispatcher backends, and expert wgrad scheduling.\nlicense: Apache-2.0\nwhen_to_use: Tuning MoE communication overlap, or tracing a MoE throughput regression to a comm-overlap config change; 'overlap_moe_expert_parallel_comm', 'MoE dispatch overlap', 'flex dispatcher', 'DeepEP overlap', 'expert wgrad scheduling'.\n---\n\n# MoE Communication Overlap\n\nFor the higher-level overview, see:\n\n- @docs\u002Ftraining\u002Fcommunication-overlap.md\n- @skills\u002Fnemo-mbridge-perf-moe-comm-overlap\u002Fcard.yaml\n\n## Quick Decision\n\nUse MoE communication overlap when:\n\n- `EP > 1`\n- token dispatch or combine time is visible in the profile\n- the run is already correct and you are now tuning throughput\n\nAvoid turning it on as an early bring-up step. It is easier to validate after\nthe dispatcher, routing mode, and recompute plan are already stable.\n\n## Enablement\n\n```python\ncfg.comm_overlap.overlap_moe_expert_parallel_comm = True\n\n# Optional: delayed wgrad for additional overlap\ncfg.comm_overlap.delay_wgrad_compute = True\n\n# IMPORTANT: disable shared expert overlap when using dispatch overlap\ncfg.model.moe_shared_expert_overlap = False\n```\n\n### Prerequisites\n\n- `expert_model_parallel_size > 1`\n- `num_moe_experts > 1`\n- `moe_token_dispatcher_type` must be `\"alltoall\"` or `\"flex\"`\n- Precision: BF16 or FP16\n- If PP is used, VPP (`virtual_pipeline_model_parallel_size`) must be set (non-`None`)\n\n### Flex dispatcher activation\n\nSetting `moe_flex_dispatcher_backend` alone does **not** activate flex dispatch.\nYou must also set `moe_token_dispatcher_type = \"flex\"`.\n\n## Recompute And CUDA Graph Interaction\n\n- Full recompute is not a good companion for the overlap path.\n- `delay_wgrad_compute` adds further constraints if CUDA-graph scopes include\n  attention or MoE-router work.\n- In practice, selective recompute is the safer pairing when overlap is enabled.\n\n## Measured Short-Run Caveat\n\nA 2026-05-18 current-main H100 x16 smoke on Qwen3 30B-A3B mock pretraining\nused `EP=16`, `alltoall`, global batch size 1024, CUDA graphs disabled, and\n`moe_permute_fusion=false` because the PyTorch 25.11 \u002F TE \u002F Triton stack failed\nin Transformer Engine fused permutation in prior bring-up.\n\nResults were directional rather than release-grade:\n\n- no EP overlap: 41.25s steady-state mean over iterations 3-8\n- EP overlap: 31.31s steady-state mean over iterations 3-8\n- EP overlap plus `delay_wgrad_compute`: 31.20s steady-state mean over\n  iterations 3-8\n\nTreat this as evidence that EP overlap can help an inter-node `alltoall` MoE\nshape when communication is exposed. It is not proof that delayed wgrad is a\nseparate win, and it does not validate the fused permutation path. An earlier\n2026-05-16 short smoke on the same shape showed the same pattern.\n\n## Code Anchors\n\n- Overlap validation: `src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fcomm_overlap.py`\n- Flex dispatcher backend: `src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fflex_dispatcher_backend.py`\n- Config: `src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fconfig.py`\n- Unit tests: `tests\u002Funit_tests\u002Ftraining\u002Ftest_comm_overlap.py`\n- DeepEP tests: `tests\u002Funit_tests\u002Ftraining\u002Ftest_deepep.py`\n\n## Pitfalls\n\n1. **Shared expert overlap conflict**: `moe_shared_expert_overlap` and\n   `overlap_moe_expert_parallel_comm` can conflict. Disable shared expert\n   overlap when using the dispatch overlap path.\n\n2. **PP without VPP**: MoE overlap requires VPP when pipeline parallelism is\n   active. Without it, the overlap scheduling cannot interleave correctly.\n\n3. **Flex != backend flag**: `moe_flex_dispatcher_backend=\"deepep\"` alone\n   does nothing if `moe_token_dispatcher_type` is still `\"alltoall\"`.\n\n4. **Conservative recipe defaults**: Most public recipes leave MoE overlap\n   disabled. You need to explicitly enable it via overrides.\n\n5. **Performance gains are workload-dependent**: overlap helps most when dispatch\n   communication is already a visible slice of step time. It is not guaranteed\n   to help every small or lightly loaded EP run.\n\n## Verification\n\nLook for overlap-related log messages during initialization. The comm overlap\nvalidation in `comm_overlap.py` will raise if prerequisites are not met, so a\nclean startup confirms the feature is active.\n\nFor a short performance-harness smoke, keep the command shape explicit and vary\nonly one overlap knob at a time:\n\n```bash\nuv run python scripts\u002Fperformance\u002Frun_script.py \\\n  -m qwen \\\n  -mr qwen3_30b_a3b \\\n  --task pretrain \\\n  -g h100 \\\n  -c bf16 \\\n  -ng 16 \\\n  -gn 8 \\\n  --max_steps 8 \\\n  --cuda_graph_impl none \\\n  --moe_flex_dispatcher_backend None \\\n  --moe_a2a_overlap false \\\n  --tokenizer_type NullTokenizer \\\n  comm_overlap.overlap_moe_expert_parallel_comm=true \\\n  comm_overlap.delay_wgrad_compute=false \\\n  model.moe_shared_expert_overlap=false\n```\n\nIf fused MoE permutation fails during bring-up, add\n`model.moe_permute_fusion=false` to separate overlap timing from runtime-stack\nvalidation, then retest with the matched production container.\n","# MoE Communication Overlap\n\nFor the higher-level overview, see:\n\n- @docs\u002Ftraining\u002Fcommunication-overlap.md\n- @skills\u002Fnemo-mbridge-perf-moe-comm-overlap\u002Fcard.yaml\n\n## Quick Decision\n\nUse MoE communication overlap when:\n\n- `EP > 1`\n- token dispatch or combine time is visible in the profile\n- the run is already correct and you are now tuning throughput\n\nAvoid turning it on as an early bring-up step. It is easier to validate after\nthe dispatcher, routing mode, and recompute plan are already stable.\n\n## Enablement\n\n```python\ncfg.comm_overlap.overlap_moe_expert_parallel_comm = True\n\n# Optional: delayed wgrad for additional overlap\ncfg.comm_overlap.delay_wgrad_compute = True\n\n# IMPORTANT: disable shared expert overlap when using dispatch overlap\ncfg.model.moe_shared_expert_overlap = False\n```\n\n### Prerequisites\n\n- `expert_model_parallel_size > 1`\n- `num_moe_experts > 1`\n- `moe_token_dispatcher_type` must be `\"alltoall\"` or `\"flex\"`\n- Precision: BF16 or FP16\n- If PP is used, VPP (`virtual_pipeline_model_parallel_size`) must be set (non-`None`)\n\n### Flex dispatcher activation\n\nSetting `moe_flex_dispatcher_backend` alone does **not** activate flex dispatch.\nYou must also set `moe_token_dispatcher_type = \"flex\"`.\n\n## Recompute And CUDA Graph Interaction\n\n- Full recompute is not a good companion for the overlap path.\n- `delay_wgrad_compute` adds further constraints if CUDA-graph scopes include\n  attention or MoE-router work.\n- In practice, selective recompute is the safer pairing when overlap is enabled.\n\n## Measured Short-Run Caveat\n\nA 2026-05-18 current-main H100 x16 smoke on Qwen3 30B-A3B mock pretraining\nused `EP=16`, `alltoall`, global batch size 1024, CUDA graphs disabled, and\n`moe_permute_fusion=false` because the PyTorch 25.11 \u002F TE \u002F Triton stack failed\nin Transformer Engine fused permutation in prior bring-up.\n\nResults were directional rather than release-grade:\n\n- no EP overlap: 41.25s steady-state mean over iterations 3-8\n- EP overlap: 31.31s steady-state mean over iterations 3-8\n- EP overlap plus `delay_wgrad_compute`: 31.20s steady-state mean over\n  iterations 3-8\n\nTreat this as evidence that EP overlap can help an inter-node `alltoall` MoE\nshape when communication is exposed. It is not proof that delayed wgrad is a\nseparate win, and it does not validate the fused permutation path. An earlier\n2026-05-16 short smoke on the same shape showed the same pattern.\n\n## Code Anchors\n\n- Overlap validation: `src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fcomm_overlap.py`\n- Flex dispatcher backend: `src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fflex_dispatcher_backend.py`\n- Config: `src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fconfig.py`\n- Unit tests: `tests\u002Funit_tests\u002Ftraining\u002Ftest_comm_overlap.py`\n- DeepEP tests: `tests\u002Funit_tests\u002Ftraining\u002Ftest_deepep.py`\n\n## Pitfalls\n\n1. **Shared expert overlap conflict**: `moe_shared_expert_overlap` and\n   `overlap_moe_expert_parallel_comm` can conflict. Disable shared expert\n   overlap when using the dispatch overlap path.\n\n2. **PP without VPP**: MoE overlap requires VPP when pipeline parallelism is\n   active. Without it, the overlap scheduling cannot interleave correctly.\n\n3. **Flex != backend flag**: `moe_flex_dispatcher_backend=\"deepep\"` alone\n   does nothing if `moe_token_dispatcher_type` is still `\"alltoall\"`.\n\n4. **Conservative recipe defaults**: Most public recipes leave MoE overlap\n   disabled. You need to explicitly enable it via overrides.\n\n5. **Performance gains are workload-dependent**: overlap helps most when dispatch\n   communication is already a visible slice of step time. It is not guaranteed\n   to help every small or lightly loaded EP run.\n\n## Verification\n\nLook for overlap-related log messages during initialization. The comm overlap\nvalidation in `comm_overlap.py` will raise if prerequisites are not met, so a\nclean startup confirms the feature is active.\n\nFor a short performance-harness smoke, keep the command shape explicit and vary\nonly one overlap knob at a time:\n\n```bash\nuv run python scripts\u002Fperformance\u002Frun_script.py \\\n  -m qwen \\\n  -mr qwen3_30b_a3b \\\n  --task pretrain \\\n  -g h100 \\\n  -c bf16 \\\n  -ng 16 \\\n  -gn 8 \\\n  --max_steps 8 \\\n  --cuda_graph_impl none \\\n  --moe_flex_dispatcher_backend None \\\n  --moe_a2a_overlap false \\\n  --tokenizer_type NullTokenizer \\\n  comm_overlap.overlap_moe_expert_parallel_comm=true \\\n  comm_overlap.delay_wgrad_compute=false \\\n  model.moe_shared_expert_overlap=false\n```\n\nIf fused MoE permutation fails during bring-up, add\n`model.moe_permute_fusion=false` to separate overlap timing from runtime-stack\nvalidation, then retest with the matched production container.\n",{"name":5,"license":15,"description":7,"when_to_use":24},"Tuning MoE communication overlap, or tracing a MoE throughput regression to a comm-overlap config change; 'overlap_moe_expert_parallel_comm', 'MoE dispatch overlap', 'flex dispatcher', 'DeepEP overlap', 'expert wgrad scheduling'.","main","be3de0244dc1a57e323d8160787ebb40fb64930e","skills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-comm-overlap","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main","https:\u002F\u002Fskills.x402.wtf","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 23:20:13",{"source":8,"skillName":5,"commands":35,"prompt":45,"zipAvailable":46},[36,39,42],{"tool":37,"command":38},"npx","npx skills add Solizardking\u002Fskillhub-main --skill nemo-mbridge-perf-moe-comm-overlap",{"tool":40,"command":41},"bunx","bunx skills add Solizardking\u002Fskillhub-main --skill nemo-mbridge-perf-moe-comm-overlap",{"tool":43,"command":44},"pnpm","pnpm dlx skills add Solizardking\u002Fskillhub-main --skill nemo-mbridge-perf-moe-comm-overlap","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-comm-overlap\u002FSKILL.md",true,[48,54,58,62,66],{"path":49,"resourceType":50,"resourceTypeDesc":51,"sizeBytes":52,"sourceHtmlUrl":53},"BENCHMARK.md","REFERENCE","参考文档",4014,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-comm-overlap\u002FBENCHMARK.md",{"path":55,"resourceType":50,"resourceTypeDesc":51,"sizeBytes":56,"sourceHtmlUrl":57},"card.yaml",2075,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-comm-overlap\u002Fcard.yaml",{"path":59,"resourceType":50,"resourceTypeDesc":51,"sizeBytes":60,"sourceHtmlUrl":61},"evals\u002Fevals.json",1678,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-comm-overlap\u002Fevals\u002Fevals.json",{"path":63,"resourceType":50,"resourceTypeDesc":51,"sizeBytes":64,"sourceHtmlUrl":65},"skill-card.md",4140,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-comm-overlap\u002Fskill-card.md",{"path":67,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":70,"sourceHtmlUrl":71},"skill.oms.sig","OTHER","其他",4829,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-moe-comm-overlap\u002Fskill.oms.sig",[73,81,88,95,102,110],{"skillNo":74,"name":75,"nameZh":6,"description":76,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":19},"01M78E0TGCAMNPS207BA3G07S7","zkrouter","Self-hosted OpenAI-compatible LLM router — smart tier-based routing, ZK-stamped routing decisions, mode overrides, free OpenRouter at install via the Birth bot. Save 60-80% on inference costs by routing to the cheapest capable model across 12+ OpenRouter models.","MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzkrouter\u002FSKILL.md",[],[],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":87,"publishedAt":19},"01M78E0R1Z8BDT6EDTHY17MTNB","zk","For custom ZK Solana programs and privacy-preserving applications to prevent double spending. Guide to integrate rent-free nullifier PDAs for double-spend prevention.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk\u002FSKILL.md",[],[],{"skillNo":89,"name":90,"nameZh":6,"description":91,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":94,"publishedAt":19},"01M78E0P3ZTZ1YWF8MD5MM23QE","zk-omni-messaging","Zero-knowledge omnichain messaging between Robinhood Chain and Solana using CheshireZkOmniMessenger (msgType 4) with nullifier anti-replay and the zk-omni-relayer service.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk-omni-messaging\u002FSKILL.md",[],[],{"skillNo":96,"name":97,"nameZh":6,"description":98,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":101,"publishedAt":19},"01M78E0JXZ2PNB1896VXAGDBWD","zerion-wallet-data","Use when the user asks to check a wallet's portfolio, token balances, DeFi positions, PnL, transaction history, or NFTs. Provides interpreted on-chain data across 40+ EVM chains and Solana via x402 pay-per-request or API key — no raw RPC decoding needed.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzerion-wallet-data\u002FSKILL.md",[],[],{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":106,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":19},"01M78E0G14FH9GRRTQYMKD5E59","youtube-clipper","YouTube 视频智能剪辑工具。下载视频和字幕，AI 分析生成精细章节（几分钟级别）， 用户选择片段后自动剪辑、翻译字幕为中英双语、烧录字幕到视频，并生成总结文案。 使用场景：当用户需要剪辑 YouTube 视频、生成短视频片段、制作双语字幕版本时。 关键词：视频剪辑、YouTube、字幕翻译、双语字幕、视频下载、clip video","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FSKILL.md",[],[],{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":114,"categories":115,"keywords":116,"publishedAt":19},"01M78E0C2ZYA99CFFDNB7ZMK5S","yield-optimization","Multi-chain yield optimization via yield.xyz (StakeKit). Discover 2,988+ yield opportunities across 75+ blockchains, build deposit\u002Fwithdrawal transactions with shell scripts, and sign them with a MoonPay wallet. Use when the user wants to earn yield, find the best lending or staking rate, enter\u002Fexit a position, or check portfolio balances.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyield-optimization\u002FSKILL.md",[],[]]