[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78D00CE7ZHXCFN23KZT95ZR":3,"skill-same:01M78D00CE7ZHXCFN23KZT95ZR":72},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":25,"commitSha":26,"skillDirectory":27,"forksCount":14,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":31,"ownerDescription":32,"updatedAt":33,"install":34,"resources":47},"01M78D00CE7ZHXCFN23KZT95ZR","nemo-mbridge-perf-megatron-fsdp",null,"Operational guide for enabling Megatron FSDP in Megatron-Bridge, including config knobs, code anchors, pitfalls, and verification.","Solizardking\u002Fskillhub-main","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786NYYXSZK1D05WM8JQPC48",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-megatron-fsdp\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: nemo-mbridge-perf-megatron-fsdp\ndescription: Operational guide for enabling Megatron FSDP in Megatron-Bridge, including config knobs, code anchors, pitfalls, and verification.\nlicense: Apache-2.0\nwhen_to_use: Using FSDP-based data parallelism instead of DDP, or tracing an OOM or regression to a FSDP config change; 'use_megatron_fsdp', 'data_parallel_sharding_strategy', 'sharded data parallel', 'Megatron FSDP'.\n---\n\n# Megatron FSDP Skill\n\nFor stable background and recommendation level, see:\n\n- @docs\u002Ftraining\u002Fmegatron-fsdp.md\n- @skills\u002Fnemo-mbridge-perf-megatron-fsdp\u002Fcard.yaml\n\n## Enablement\n\nMinimal Megatron FSDP override in Bridge:\n\n```python\ncfg.dist.use_megatron_fsdp = True\ncfg.ddp.use_megatron_fsdp = True\ncfg.ddp.data_parallel_sharding_strategy = \"optim_grads_params\"\ncfg.ddp.average_in_collective = False\ncfg.checkpoint.ckpt_format = \"fsdp_dtensor\"\n```\n\nExample recipe fixup:\n\n```python\ncfg = llama3_8b_pretrain_config()\ncfg.dist.use_megatron_fsdp = True\ncfg.ddp.use_megatron_fsdp = True\ncfg.ddp.data_parallel_sharding_strategy = \"optim_grads_params\"\ncfg.ddp.average_in_collective = False\ncfg.checkpoint.ckpt_format = \"fsdp_dtensor\"\ncfg.checkpoint.save = \"\u002Ftmp\u002Ffsdp_ckpts\"\ncfg.checkpoint.load = None\n```\n\nPerformance harness note:\n\n```bash\npython scripts\u002Fperformance\u002Flaunch.py --use_megatron_fsdp true\n```\n\n## Code Anchors\n\nBridge config definition:\n\n```148:154:src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fconfig.py\nuse_megatron_fsdp: bool = False\n\"\"\"Use Megatron's Fully Sharded Data Parallel. Cannot be used together with use_torch_fsdp2.\"\"\"\n\nuse_torch_fsdp2: bool = False\n\"\"\"Use the torch FSDP2 implementation. FSDP2 is not currently working with Pipeline Parallel.\nIt is still not in a stable release stage, and may therefore contain bugs or other\npotential issues.\"\"\"\n```\n\nBridge validation:\n\n```1533:1578:src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fconfig.py\nif self.dist.use_megatron_fsdp and self.dist.use_torch_fsdp2:\n    raise ValueError(...)\n...\nassert not self.dist.use_tp_pp_dp_mapping, \"use_tp_pp_dp_mapping is not supported with Megatron FSDP\"\n...\nassert self.checkpoint.ckpt_format == \"fsdp_dtensor\", (\n    \"Megatron FSDP only supports fsdp_dtensor checkpoint format\"\n)\n```\n\nRuntime wrapper selection:\n\n```217:243:src\u002Fmegatron\u002Fbridge\u002Fmodels\u002Fcommon\u002Funimodal.py\nif use_megatron_fsdp:\n    DP = FullyShardedDataParallel\nelif use_torch_fsdp2:\n    DP = TorchFullyShardedDataParallel\nelse:\n    DP = DistributedDataParallel\n...\nDP(\n    config=get_model_config(model_chunk),\n    ddp_config=ddp_config,\n    module=model_chunk,\n    ...\n    pg_collection=pg_collection,\n)\n```\n\nPerf harness overrides:\n\n```74:98:scripts\u002Fperformance\u002Futils\u002Foverrides.py\nrecipe.ddp.use_megatron_fsdp = True\nrecipe.ddp.data_parallel_sharding_strategy = \"optim_grads_params\"\nrecipe.ddp.keep_fp8_transpose_cache = False\nrecipe.ddp.average_in_collective = False\n...\nrecipe.checkpoint.load = None\n```\n\n## Pitfalls\n\n1. Public recipes often expose `use_megatron_fsdp` but still default to `ckpt_format=\"torch_dist\"`. If save\u002Fload is enabled, switch to `fsdp_dtensor`.\n2. `use_torch_fsdp2` exists, but on the validated branch Bridge still fails before training because `_ddp_wrap` passes `pg_collection`.\n3. CPU offloading is only valid when `pipeline_model_parallel_size == 1` and activation recomputation is disabled.\n4. Upstream warns that FSDP and TP\u002FCP can want different `CUDA_DEVICE_MAX_CONNECTIONS` settings on Hopper and earlier.\n5. Megatron FSDP and FSDP2 are mutually exclusive.\n\n## Verification\n\nUse the existing 2-GPU functional smoke test:\n\n```bash\nCUDA_VISIBLE_DEVICES=0,1 uv run python -m torch.distributed.run --nproc_per_node=2 \\\n  -m pytest tests\u002Ffunctional_tests\u002Ftraining\u002Ftest_megatron_fsdp.py::TestMegatronFSDP::test_fsdp_pretrain_basic -v -s\n```\n\nSuccess criteria:\n\n- Pytest reports `1 passed`\n- The log shows finite loss at the last iteration\n- The run finishes without a checkpoint format assertion\n","# Megatron FSDP Skill\n\nFor stable background and recommendation level, see:\n\n- @docs\u002Ftraining\u002Fmegatron-fsdp.md\n- @skills\u002Fnemo-mbridge-perf-megatron-fsdp\u002Fcard.yaml\n\n## Enablement\n\nMinimal Megatron FSDP override in Bridge:\n\n```python\ncfg.dist.use_megatron_fsdp = True\ncfg.ddp.use_megatron_fsdp = True\ncfg.ddp.data_parallel_sharding_strategy = \"optim_grads_params\"\ncfg.ddp.average_in_collective = False\ncfg.checkpoint.ckpt_format = \"fsdp_dtensor\"\n```\n\nExample recipe fixup:\n\n```python\ncfg = llama3_8b_pretrain_config()\ncfg.dist.use_megatron_fsdp = True\ncfg.ddp.use_megatron_fsdp = True\ncfg.ddp.data_parallel_sharding_strategy = \"optim_grads_params\"\ncfg.ddp.average_in_collective = False\ncfg.checkpoint.ckpt_format = \"fsdp_dtensor\"\ncfg.checkpoint.save = \"\u002Ftmp\u002Ffsdp_ckpts\"\ncfg.checkpoint.load = None\n```\n\nPerformance harness note:\n\n```bash\npython scripts\u002Fperformance\u002Flaunch.py --use_megatron_fsdp true\n```\n\n## Code Anchors\n\nBridge config definition:\n\n```148:154:src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fconfig.py\nuse_megatron_fsdp: bool = False\n\"\"\"Use Megatron's Fully Sharded Data Parallel. Cannot be used together with use_torch_fsdp2.\"\"\"\n\nuse_torch_fsdp2: bool = False\n\"\"\"Use the torch FSDP2 implementation. FSDP2 is not currently working with Pipeline Parallel.\nIt is still not in a stable release stage, and may therefore contain bugs or other\npotential issues.\"\"\"\n```\n\nBridge validation:\n\n```1533:1578:src\u002Fmegatron\u002Fbridge\u002Ftraining\u002Fconfig.py\nif self.dist.use_megatron_fsdp and self.dist.use_torch_fsdp2:\n    raise ValueError(...)\n...\nassert not self.dist.use_tp_pp_dp_mapping, \"use_tp_pp_dp_mapping is not supported with Megatron FSDP\"\n...\nassert self.checkpoint.ckpt_format == \"fsdp_dtensor\", (\n    \"Megatron FSDP only supports fsdp_dtensor checkpoint format\"\n)\n```\n\nRuntime wrapper selection:\n\n```217:243:src\u002Fmegatron\u002Fbridge\u002Fmodels\u002Fcommon\u002Funimodal.py\nif use_megatron_fsdp:\n    DP = FullyShardedDataParallel\nelif use_torch_fsdp2:\n    DP = TorchFullyShardedDataParallel\nelse:\n    DP = DistributedDataParallel\n...\nDP(\n    config=get_model_config(model_chunk),\n    ddp_config=ddp_config,\n    module=model_chunk,\n    ...\n    pg_collection=pg_collection,\n)\n```\n\nPerf harness overrides:\n\n```74:98:scripts\u002Fperformance\u002Futils\u002Foverrides.py\nrecipe.ddp.use_megatron_fsdp = True\nrecipe.ddp.data_parallel_sharding_strategy = \"optim_grads_params\"\nrecipe.ddp.keep_fp8_transpose_cache = False\nrecipe.ddp.average_in_collective = False\n...\nrecipe.checkpoint.load = None\n```\n\n## Pitfalls\n\n1. Public recipes often expose `use_megatron_fsdp` but still default to `ckpt_format=\"torch_dist\"`. If save\u002Fload is enabled, switch to `fsdp_dtensor`.\n2. `use_torch_fsdp2` exists, but on the validated branch Bridge still fails before training because `_ddp_wrap` passes `pg_collection`.\n3. CPU offloading is only valid when `pipeline_model_parallel_size == 1` and activation recomputation is disabled.\n4. Upstream warns that FSDP and TP\u002FCP can want different `CUDA_DEVICE_MAX_CONNECTIONS` settings on Hopper and earlier.\n5. Megatron FSDP and FSDP2 are mutually exclusive.\n\n## Verification\n\nUse the existing 2-GPU functional smoke test:\n\n```bash\nCUDA_VISIBLE_DEVICES=0,1 uv run python -m torch.distributed.run --nproc_per_node=2 \\\n  -m pytest tests\u002Ffunctional_tests\u002Ftraining\u002Ftest_megatron_fsdp.py::TestMegatronFSDP::test_fsdp_pretrain_basic -v -s\n```\n\nSuccess criteria:\n\n- Pytest reports `1 passed`\n- The log shows finite loss at the last iteration\n- The run finishes without a checkpoint format assertion\n",{"name":5,"license":15,"description":7,"when_to_use":24},"Using FSDP-based data parallelism instead of DDP, or tracing an OOM or regression to a FSDP config change; 'use_megatron_fsdp', 'data_parallel_sharding_strategy', 'sharded data parallel', 'Megatron FSDP'.","main","be3de0244dc1a57e323d8160787ebb40fb64930e","skills\u002Fnvidia\u002Fnemo-mbridge-perf-megatron-fsdp","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main","https:\u002F\u002Fskills.x402.wtf","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 23:20:13",{"source":8,"skillName":5,"commands":35,"prompt":45,"zipAvailable":46},[36,39,42],{"tool":37,"command":38},"npx","npx skills add Solizardking\u002Fskillhub-main --skill nemo-mbridge-perf-megatron-fsdp",{"tool":40,"command":41},"bunx","bunx skills add Solizardking\u002Fskillhub-main --skill nemo-mbridge-perf-megatron-fsdp",{"tool":43,"command":44},"pnpm","pnpm dlx skills add Solizardking\u002Fskillhub-main --skill nemo-mbridge-perf-megatron-fsdp","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-megatron-fsdp\u002FSKILL.md",true,[48,54,58,62,66],{"path":49,"resourceType":50,"resourceTypeDesc":51,"sizeBytes":52,"sourceHtmlUrl":53},"BENCHMARK.md","REFERENCE","参考文档",4331,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-megatron-fsdp\u002FBENCHMARK.md",{"path":55,"resourceType":50,"resourceTypeDesc":51,"sizeBytes":56,"sourceHtmlUrl":57},"card.yaml",2108,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-megatron-fsdp\u002Fcard.yaml",{"path":59,"resourceType":50,"resourceTypeDesc":51,"sizeBytes":60,"sourceHtmlUrl":61},"evals\u002Fevals.json",1434,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-megatron-fsdp\u002Fevals\u002Fevals.json",{"path":63,"resourceType":50,"resourceTypeDesc":51,"sizeBytes":64,"sourceHtmlUrl":65},"skill-card.md",3760,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-megatron-fsdp\u002Fskill-card.md",{"path":67,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":70,"sourceHtmlUrl":71},"skill.oms.sig","OTHER","其他",4825,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fnvidia\u002Fnemo-mbridge-perf-megatron-fsdp\u002Fskill.oms.sig",[73,81,88,95,102,110],{"skillNo":74,"name":75,"nameZh":6,"description":76,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":19},"01M78E0TGCAMNPS207BA3G07S7","zkrouter","Self-hosted OpenAI-compatible LLM router — smart tier-based routing, ZK-stamped routing decisions, mode overrides, free OpenRouter at install via the Birth bot. Save 60-80% on inference costs by routing to the cheapest capable model across 12+ OpenRouter models.","MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzkrouter\u002FSKILL.md",[],[],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":87,"publishedAt":19},"01M78E0R1Z8BDT6EDTHY17MTNB","zk","For custom ZK Solana programs and privacy-preserving applications to prevent double spending. Guide to integrate rent-free nullifier PDAs for double-spend prevention.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk\u002FSKILL.md",[],[],{"skillNo":89,"name":90,"nameZh":6,"description":91,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":94,"publishedAt":19},"01M78E0P3ZTZ1YWF8MD5MM23QE","zk-omni-messaging","Zero-knowledge omnichain messaging between Robinhood Chain and Solana using CheshireZkOmniMessenger (msgType 4) with nullifier anti-replay and the zk-omni-relayer service.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzk-omni-messaging\u002FSKILL.md",[],[],{"skillNo":96,"name":97,"nameZh":6,"description":98,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":101,"publishedAt":19},"01M78E0JXZ2PNB1896VXAGDBWD","zerion-wallet-data","Use when the user asks to check a wallet's portfolio, token balances, DeFi positions, PnL, transaction history, or NFTs. Provides interpreted on-chain data across 40+ EVM chains and Solana via x402 pay-per-request or API key — no raw RPC decoding needed.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fzerion-wallet-data\u002FSKILL.md",[],[],{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":106,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":19},"01M78E0G14FH9GRRTQYMKD5E59","youtube-clipper","YouTube 视频智能剪辑工具。下载视频和字幕，AI 分析生成精细章节（几分钟级别）， 用户选择片段后自动剪辑、翻译字幕为中英双语、烧录字幕到视频，并生成总结文案。 使用场景：当用户需要剪辑 YouTube 视频、生成短视频片段、制作双语字幕版本时。 关键词：视频剪辑、YouTube、字幕翻译、双语字幕、视频下载、clip video","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyoutube-clipper\u002FSKILL.md",[],[],{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":77,"licenseFileUrl":6,"sourceHtmlUrl":114,"categories":115,"keywords":116,"publishedAt":19},"01M78E0C2ZYA99CFFDNB7ZMK5S","yield-optimization","Multi-chain yield optimization via yield.xyz (StakeKit). Discover 2,988+ yield opportunities across 75+ blockchains, build deposit\u002Fwithdrawal transactions with shell scripts, and sign them with a MoonPay wallet. Use when the user wants to earn yield, find the best lending or staking rate, enter\u002Fexit a position, or check portfolio balances.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Fskillhub-main\u002Fblob\u002Fmain\u002Fskills\u002Fyield-optimization\u002FSKILL.md",[],[]]