[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M788BXFGSQNKAN5SAJSVFT8Y":3,"skill-same:01M788BXFGSQNKAN5SAJSVFT8Y":96},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":32,"commitSha":33,"skillDirectory":34,"forksCount":14,"repositoryHtmlUrl":35,"repositoryHomepageUrl":6,"ownerHtmlUrl":36,"ownerBlogUrl":37,"ownerDescription":38,"updatedAt":39,"install":40,"resources":53},"01M788BXFGSQNKAN5SAJSVFT8Y","tao-run-on-slurm",null,"Remote SLURM GPU cluster execution over SSH with sbatch\u002Fsrun, Pyxis\u002FEnroot containers, and Lustre-backed results. Use when running TAO training\u002Feval\u002Finference jobs on an on-prem or DGX SLURM cluster. Trigger phrases include \"run on SLURM\", \"submit sbatch\", \"DGX SLURM cluster\", \"Pyxis\u002FEnroot container\", \"Lustre dataset\".","Solizardking\u002Ftrain2earn","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786KDC5C3NPE4W44C5641EZ",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: tao-run-on-slurm\ndescription: Remote SLURM GPU cluster execution over SSH with sbatch\u002Fsrun, Pyxis\u002FEnroot containers, and Lustre-backed\n  results. Use when running TAO training\u002Feval\u002Finference jobs on an on-prem or DGX SLURM cluster. Trigger phrases include\n  \"run on SLURM\", \"submit sbatch\", \"DGX SLURM cluster\", \"Pyxis\u002FEnroot container\", \"Lustre dataset\".\nlicense: Apache-2.0\ncompatibility: Requires SSH access to a SLURM login node (passwordless via key auth) and SLURM_USER + SLURM_HOSTNAME env vars.\n  The TAO SDK with the slurm extra (pip install 'nvidia-tao-sdk[slurm]') is needed only if you want Job handles, S3 I\u002FO wrapping,\n  or run-folder durability via ActionWorkflow.\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash\ntags:\n- platform\n- slurm\n---\n\n# SLURM\n\nRemote GPU compute platform for clusters managed by SLURM. Jobs are submitted\nfrom the TAO service or SDK host to a login node over SSH, staged on a shared\nfilesystem, submitted with `sbatch`, and executed with `srun` container support.\n\n## When to use\n\nUse SLURM when the user has access to a managed GPU cluster, shared Lustre\nstorage, and scheduler-owned GPU allocation. Do not use SLURM for local files\nthat exist only on the agent machine; data and outputs must be reachable from\nthe cluster.\n\n## Preflight + SSH\n\nConfirm `SLURM_USER` and `SLURM_HOSTNAME` are exported and passwordless SSH to a\nlogin host works (`ssh -o BatchMode=yes`). Optionally install the TAO SDK\nwrapper for Job handles + S3 wrapping (`nvidia-tao-sdk[slurm]`, on public PyPI).\nFor private `nvcr.io` images, install `~\u002F.config\u002Fenroot\u002F.credentials` on the\ncluster once per (cluster, user): Pyxis\u002FEnroot does not read `NGC_KEY` from the\njob env, and without persistent credentials, auth-gated pulls fail with \"Could\nnot process JSON input\" at job startup. Install it via the `printf | ssh`\nheredoc so the `NGC_KEY` value never lands in shell history, intermediate files,\nor chat output; never `cat`\u002F`echo` the value.\n\nIf a preflight check fails, the agent prompts the user to authorize the\ninstall\u002Ffix via Bash. Pip-installable Python requirements are the exception:\ninstall them automatically, then rerun preflight.\n\nSee `references\u002Fslurm-ssh-credentials.md` for the full preflight script, the\nenroot-credentials heredoc, prerequisite key setup (keypair, `ssh-copy-id`,\n`known_hosts`, container key mounts, 2FA handling), and the SSH failure\nremediation prompt.\n\n## Storage\n\nUse shared-filesystem URIs, not local or `file:\u002F\u002F` paths; `tao-core` rejects\nlocal\u002Ffile paths for remote backends.\n\n- `lustre:\u002F\u002F\u002Fabsolute\u002Fpath` for user-provided datasets on Lustre.\n- `slurm:\u002F\u002F` paths may appear in microservices metadata and are converted to\n  Lustre paths before the container starts.\n\nAccept either dataset roots (model skills map them to required files) or direct\nspec-key paths. After SSH succeeds and before generating scripts, `test -e` each\nrequired dataset path from the login host; if it fails, stop and ask for\ncorrected paths or staged data rather than producing scripts that fail in the\nfirst training job. See `references\u002Fslurm-ssh-credentials.md` for root vs.\ndirect-spec modes, backend details, and the results-dir default.\n\n## Container execution\n\n`tao-core` runs TAO containers through Pyxis\u002FEnroot:\n\n1. Stage compact JSON files for specs, environment, and cloud metadata under\n   `\u003Cjob_dir>\u002Fspecs`, `\u003Cjob_dir>\u002Fenv`, and `\u003Cjob_dir>\u002Fmeta`.\n2. Optionally convert the Docker image to a cached SQSH image with\n   `srun -n1 -p \u003Cconversion_partition> enroot import`.\n3. Write an sbatch script under `\u003Cjob_dir>\u002Fsbatch\u002Fjob_\u003Cjob_id>.sbatch`.\n4. Submit `sbatch --export=ALL \u003Cscript>`.\n5. Run the container with `srun --container-image=\u003Cimage> --container-mounts=\u002Flustre`.\n\nAccepted image formats: `\u002Fpath\u002Fto\u002Fimage.sqsh`, `registry#image:tag`,\n`docker:\u002F\u002Fregistry#image:tag`, and ordinary `registry\u002Fimage:tag` (converted to\nPyxis form when needed). SQSH conversion is cached by image name; for `:latest`\nimages the cached SQSH is reused unless `force_reconvert_latest` is enabled.\n\n## Monitoring and cancellation\n\n- Scheduler status comes from the stored SLURM job id via `squeue`\u002F`sacct`;\n  TAO terminal status comes from `status.json` in the shared results folder.\n- While chat monitoring is enabled, keep polling at the requested interval for\n  any non-terminal job (`PENDING`, `RUNNING`, or otherwise). Do not stop after a\n  fixed elapsed time such as 30 minutes; long queue waits are normal on shared\n  GPU partitions.\n- Do not send a final response for a non-terminal SLURM job when chat\n  monitoring is enabled. A final response is a detach action; use it only if the\n  user asked to detach\u002Fstop or the job reached terminal state.\n- Logs are read over SSH from\n  `\u003Cjob_dir>\u002Fslurm-logs\u002F\u003Cslurm_job_name>-\u003Cslurm_job_id>\u002Fmain.out` and `.err`.\n- Cancel by looking up `backend_details.slurm_metadata.slurm_job_id` and running\n  `scancel \u003Cslurm_job_id>` over SSH. Treat missing or already terminated jobs as\n  successful cancellation.\n\nStatus mapping:\n\n- `PENDING` -> `Pending`\n- `RUNNING` or `COMPLETING` -> `Running`\n- `COMPLETED` -> check `status.json`\n- `FAILED`, `BOOT_FAIL`, `DEADLINE`, `OUT_OF_MEMORY`, `NODE_FAIL` -> retry if\n  logs match retriable infrastructure patterns, otherwise `Error`\n- `CANCELLED`, `PREEMPTED`, `REVOKED` -> `Canceled`\n- `TIMEOUT` -> `Error`\n- `SUSPENDED`, `STOPPED` -> `Paused`\n\n## Required inputs\n\nAsk for these in the SLURM intake; see `references\u002Fslurm-ssh-credentials.md`\nfor the full credential list, microservices schema keys, and defaults.\n\n- **SLURM_USER** (required): SSH username for the login node.\n- **SLURM_HOSTNAME** (required): Comma-separated login hostnames for failover.\n- **SLURM_PARTITION** (required): Partition list for GPU submission. Packaged\n  default `polar,polar3,polar4,grizzly`, treated as 4-hour queues.\n- **SSH_KEY_PATH** (preferred, expected before launch): private key for\n  non-interactive public-key auth. Ask for this first in remediation; prefer it\n  over the `SSH_AUTH_SOCK` agent-socket fallback.\n- **SLURM_BASE_RESULTS_DIR** (optional): base shared-filesystem path; default\n  `\u002Flustre\u002Ffsw\u002Fportfolios\u002Fedgeai\u002Fusers\u002F\u003Cyour-dir>` (your per-user Lustre dir).\n- **SLURM_ACCOUNT** (usually required by site policy): account for `#SBATCH --account`.\n\nDo not ask for `SLURM_ACCOUNT` or `SLURM_BASE_RESULTS_DIR` in the initial\nintake unless the user says their site requires an account, wants a custom\nresults root, or the workflow cannot proceed without overriding defaults.\n\n## Resource defaults\n\nDefaults from `tao-core`:\n\n- `num_nodes`: 1\n- `num_gpus`: 4\n- `max_num_gpus_per_node`: 8\n- `cpus_per_task`: 16\n- `time_hours`: 4\n- `timeout_hours`: 3.8\n- `max_time_hours`: 4\n- `container_mounts`: `\u002Flustre`\n- `use_requeue`: true\n- `use_sqsh`: true\n\nWhen generating launchers or wrapper scripts for SLURM, set the wall-time\ndefaults explicitly from the packaged platform resource defaults:\n\n```bash\nexport SLURM_TIME_HOURS=\"${SLURM_TIME_HOURS:-4}\"\nexport SLURM_TIMEOUT_HOURS=\"${SLURM_TIMEOUT_HOURS:-3.8}\"\n```\n\nDo not default to 12 hours on SLURM. If the user supplies a longer\n`SLURM_TIME_HOURS`, verify that the selected partition supports it before\nsubmitting. For the packaged default partition list\n`polar,polar3,polar4,grizzly`, reject requests above 4 hours and ask for a\ndifferent partition only if the user actually wants a longer wall time.\n\nWhen `num_gpus` is greater than or equal to `max_num_gpus_per_node`, the\nhandler treats the request as exclusive per node and computes additional nodes\nfrom total GPU count when necessary.\n\n## Multi-node, SDK, and retries\n\nFor multi-node jobs (`num_nodes > 1`), the SDK builds the sbatch directives and\nexports the PyTorch-distributed rendezvous env vars automatically: `WORLD_SIZE`,\n`NUM_GPU_PER_NODE`, `NODE_RANK`, `MASTER_ADDR`, and `MASTER_PORT` (29500).\nTAO entrypoints read `WORLD_SIZE` + `NUM_GPU_PER_NODE` and build torchrun\ninternally. Cosmos-RL has special multi-node role handling for controller,\npolicy, and rollout workers.\n\n**Use Lustre, not S3, for SLURM job inputs.** The GPU allocation starts the\nmoment the job is dispatched, so a long `s3:\u002F\u002F` download at the top of the\nscript burns the allocation, can get the job killed for GPU-idle, and is billed\neither way. Stage training data on the shared filesystem first and reference it\nas `lustre:\u002F\u002F\u002F...`. S3\u002FHF\u002FNGC pre-fetch is fine for small auxiliary inputs\n(checkpoints, configs), not training datasets. K8s\u002FBrev do not share this\nscheduler-idle constraint.\n\nAuto-retry of infrastructure failures (`NODE_FAIL`, `BOOT_FAIL`, NCCL transport\ntimeouts, CUDA driver init failures, GPU\u002FIB link-down, OOM-killer node reaping,\nXid errors) is automatic in the SDK, with a stable user-facing `Job.id` across\nretries. Plain training failures surface immediately so a broken spec does not\nconsume the retry budget. `#SBATCH --requeue` is enabled by default via\n`SLURM_USE_REQUEUE=true`.\n\nSee `references\u002Fslurm-container-execution.md` for the full multi-node\nenv-var\u002Fsbatch directive detail and table, cluster requirements, the optional\nTAO SDK path (`SlurmSDK`, `build_entrypoint`, `ActionWorkflow`) with code, the\nLustre-not-S3 rule in full, and the failure-mode checklist;\n`references\u002Fslurm-execution-sdk.md` covers the `MAX_JOB_RETRIES` retry budget.\nWhen the SDK is in scope, read `tao-skill-bank:tao-run-platform` for the\n`SlurmSDK` kwarg reference.\n\n## References\n\n- `references\u002Fslurm-ssh-credentials.md` — preflight script, SSH\u002Fkey setup,\n  enroot credentials, full credential list, backend details, storage rules,\n  SSH remediation prompt.\n- `references\u002Fslurm-container-execution.md` — container execution steps,\n  monitoring, status mapping, cancellation, multi-node detail, SDK use,\n  Lustre-not-S3, auto-retry, failure modes.\n- `references\u002Fslurm-preflight-storage.md` — extended preflight\u002Fstorage notes.\n- `references\u002Fslurm-execution-sdk.md` — extended execution\u002FSDK notes.\n- `references\u002Fdetailed-guide.md` — navigation map for the split references.\n","# SLURM\n\nRemote GPU compute platform for clusters managed by SLURM. Jobs are submitted\nfrom the TAO service or SDK host to a login node over SSH, staged on a shared\nfilesystem, submitted with `sbatch`, and executed with `srun` container support.\n\n## When to use\n\nUse SLURM when the user has access to a managed GPU cluster, shared Lustre\nstorage, and scheduler-owned GPU allocation. Do not use SLURM for local files\nthat exist only on the agent machine; data and outputs must be reachable from\nthe cluster.\n\n## Preflight + SSH\n\nConfirm `SLURM_USER` and `SLURM_HOSTNAME` are exported and passwordless SSH to a\nlogin host works (`ssh -o BatchMode=yes`). Optionally install the TAO SDK\nwrapper for Job handles + S3 wrapping (`nvidia-tao-sdk[slurm]`, on public PyPI).\nFor private `nvcr.io` images, install `~\u002F.config\u002Fenroot\u002F.credentials` on the\ncluster once per (cluster, user): Pyxis\u002FEnroot does not read `NGC_KEY` from the\njob env, and without persistent credentials, auth-gated pulls fail with \"Could\nnot process JSON input\" at job startup. Install it via the `printf | ssh`\nheredoc so the `NGC_KEY` value never lands in shell history, intermediate files,\nor chat output; never `cat`\u002F`echo` the value.\n\nIf a preflight check fails, the agent prompts the user to authorize the\ninstall\u002Ffix via Bash. Pip-installable Python requirements are the exception:\ninstall them automatically, then rerun preflight.\n\nSee `references\u002Fslurm-ssh-credentials.md` for the full preflight script, the\nenroot-credentials heredoc, prerequisite key setup (keypair, `ssh-copy-id`,\n`known_hosts`, container key mounts, 2FA handling), and the SSH failure\nremediation prompt.\n\n## Storage\n\nUse shared-filesystem URIs, not local or `file:\u002F\u002F` paths; `tao-core` rejects\nlocal\u002Ffile paths for remote backends.\n\n- `lustre:\u002F\u002F\u002Fabsolute\u002Fpath` for user-provided datasets on Lustre.\n- `slurm:\u002F\u002F` paths may appear in microservices metadata and are converted to\n  Lustre paths before the container starts.\n\nAccept either dataset roots (model skills map them to required files) or direct\nspec-key paths. After SSH succeeds and before generating scripts, `test -e` each\nrequired dataset path from the login host; if it fails, stop and ask for\ncorrected paths or staged data rather than producing scripts that fail in the\nfirst training job. See `references\u002Fslurm-ssh-credentials.md` for root vs.\ndirect-spec modes, backend details, and the results-dir default.\n\n## Container execution\n\n`tao-core` runs TAO containers through Pyxis\u002FEnroot:\n\n1. Stage compact JSON files for specs, environment, and cloud metadata under\n   `\u003Cjob_dir>\u002Fspecs`, `\u003Cjob_dir>\u002Fenv`, and `\u003Cjob_dir>\u002Fmeta`.\n2. Optionally convert the Docker image to a cached SQSH image with\n   `srun -n1 -p \u003Cconversion_partition> enroot import`.\n3. Write an sbatch script under `\u003Cjob_dir>\u002Fsbatch\u002Fjob_\u003Cjob_id>.sbatch`.\n4. Submit `sbatch --export=ALL \u003Cscript>`.\n5. Run the container with `srun --container-image=\u003Cimage> --container-mounts=\u002Flustre`.\n\nAccepted image formats: `\u002Fpath\u002Fto\u002Fimage.sqsh`, `registry#image:tag`,\n`docker:\u002F\u002Fregistry#image:tag`, and ordinary `registry\u002Fimage:tag` (converted to\nPyxis form when needed). SQSH conversion is cached by image name; for `:latest`\nimages the cached SQSH is reused unless `force_reconvert_latest` is enabled.\n\n## Monitoring and cancellation\n\n- Scheduler status comes from the stored SLURM job id via `squeue`\u002F`sacct`;\n  TAO terminal status comes from `status.json` in the shared results folder.\n- While chat monitoring is enabled, keep polling at the requested interval for\n  any non-terminal job (`PENDING`, `RUNNING`, or otherwise). Do not stop after a\n  fixed elapsed time such as 30 minutes; long queue waits are normal on shared\n  GPU partitions.\n- Do not send a final response for a non-terminal SLURM job when chat\n  monitoring is enabled. A final response is a detach action; use it only if the\n  user asked to detach\u002Fstop or the job reached terminal state.\n- Logs are read over SSH from\n  `\u003Cjob_dir>\u002Fslurm-logs\u002F\u003Cslurm_job_name>-\u003Cslurm_job_id>\u002Fmain.out` and `.err`.\n- Cancel by looking up `backend_details.slurm_metadata.slurm_job_id` and running\n  `scancel \u003Cslurm_job_id>` over SSH. Treat missing or already terminated jobs as\n  successful cancellation.\n\nStatus mapping:\n\n- `PENDING` -> `Pending`\n- `RUNNING` or `COMPLETING` -> `Running`\n- `COMPLETED` -> check `status.json`\n- `FAILED`, `BOOT_FAIL`, `DEADLINE`, `OUT_OF_MEMORY`, `NODE_FAIL` -> retry if\n  logs match retriable infrastructure patterns, otherwise `Error`\n- `CANCELLED`, `PREEMPTED`, `REVOKED` -> `Canceled`\n- `TIMEOUT` -> `Error`\n- `SUSPENDED`, `STOPPED` -> `Paused`\n\n## Required inputs\n\nAsk for these in the SLURM intake; see `references\u002Fslurm-ssh-credentials.md`\nfor the full credential list, microservices schema keys, and defaults.\n\n- **SLURM_USER** (required): SSH username for the login node.\n- **SLURM_HOSTNAME** (required): Comma-separated login hostnames for failover.\n- **SLURM_PARTITION** (required): Partition list for GPU submission. Packaged\n  default `polar,polar3,polar4,grizzly`, treated as 4-hour queues.\n- **SSH_KEY_PATH** (preferred, expected before launch): private key for\n  non-interactive public-key auth. Ask for this first in remediation; prefer it\n  over the `SSH_AUTH_SOCK` agent-socket fallback.\n- **SLURM_BASE_RESULTS_DIR** (optional): base shared-filesystem path; default\n  `\u002Flustre\u002Ffsw\u002Fportfolios\u002Fedgeai\u002Fusers\u002F\u003Cyour-dir>` (your per-user Lustre dir).\n- **SLURM_ACCOUNT** (usually required by site policy): account for `#SBATCH --account`.\n\nDo not ask for `SLURM_ACCOUNT` or `SLURM_BASE_RESULTS_DIR` in the initial\nintake unless the user says their site requires an account, wants a custom\nresults root, or the workflow cannot proceed without overriding defaults.\n\n## Resource defaults\n\nDefaults from `tao-core`:\n\n- `num_nodes`: 1\n- `num_gpus`: 4\n- `max_num_gpus_per_node`: 8\n- `cpus_per_task`: 16\n- `time_hours`: 4\n- `timeout_hours`: 3.8\n- `max_time_hours`: 4\n- `container_mounts`: `\u002Flustre`\n- `use_requeue`: true\n- `use_sqsh`: true\n\nWhen generating launchers or wrapper scripts for SLURM, set the wall-time\ndefaults explicitly from the packaged platform resource defaults:\n\n```bash\nexport SLURM_TIME_HOURS=\"${SLURM_TIME_HOURS:-4}\"\nexport SLURM_TIMEOUT_HOURS=\"${SLURM_TIMEOUT_HOURS:-3.8}\"\n```\n\nDo not default to 12 hours on SLURM. If the user supplies a longer\n`SLURM_TIME_HOURS`, verify that the selected partition supports it before\nsubmitting. For the packaged default partition list\n`polar,polar3,polar4,grizzly`, reject requests above 4 hours and ask for a\ndifferent partition only if the user actually wants a longer wall time.\n\nWhen `num_gpus` is greater than or equal to `max_num_gpus_per_node`, the\nhandler treats the request as exclusive per node and computes additional nodes\nfrom total GPU count when necessary.\n\n## Multi-node, SDK, and retries\n\nFor multi-node jobs (`num_nodes > 1`), the SDK builds the sbatch directives and\nexports the PyTorch-distributed rendezvous env vars automatically: `WORLD_SIZE`,\n`NUM_GPU_PER_NODE`, `NODE_RANK`, `MASTER_ADDR`, and `MASTER_PORT` (29500).\nTAO entrypoints read `WORLD_SIZE` + `NUM_GPU_PER_NODE` and build torchrun\ninternally. Cosmos-RL has special multi-node role handling for controller,\npolicy, and rollout workers.\n\n**Use Lustre, not S3, for SLURM job inputs.** The GPU allocation starts the\nmoment the job is dispatched, so a long `s3:\u002F\u002F` download at the top of the\nscript burns the allocation, can get the job killed for GPU-idle, and is billed\neither way. Stage training data on the shared filesystem first and reference it\nas `lustre:\u002F\u002F\u002F...`. S3\u002FHF\u002FNGC pre-fetch is fine for small auxiliary inputs\n(checkpoints, configs), not training datasets. K8s\u002FBrev do not share this\nscheduler-idle constraint.\n\nAuto-retry of infrastructure failures (`NODE_FAIL`, `BOOT_FAIL`, NCCL transport\ntimeouts, CUDA driver init failures, GPU\u002FIB link-down, OOM-killer node reaping,\nXid errors) is automatic in the SDK, with a stable user-facing `Job.id` across\nretries. Plain training failures surface immediately so a broken spec does not\nconsume the retry budget. `#SBATCH --requeue` is enabled by default via\n`SLURM_USE_REQUEUE=true`.\n\nSee `references\u002Fslurm-container-execution.md` for the full multi-node\nenv-var\u002Fsbatch directive detail and table, cluster requirements, the optional\nTAO SDK path (`SlurmSDK`, `build_entrypoint`, `ActionWorkflow`) with code, the\nLustre-not-S3 rule in full, and the failure-mode checklist;\n`references\u002Fslurm-execution-sdk.md` covers the `MAX_JOB_RETRIES` retry budget.\nWhen the SDK is in scope, read `tao-skill-bank:tao-run-platform` for the\n`SlurmSDK` kwarg reference.\n\n## References\n\n- `references\u002Fslurm-ssh-credentials.md` — preflight script, SSH\u002Fkey setup,\n  enroot credentials, full credential list, backend details, storage rules,\n  SSH remediation prompt.\n- `references\u002Fslurm-container-execution.md` — container execution steps,\n  monitoring, status mapping, cancellation, multi-node detail, SDK use,\n  Lustre-not-S3, auto-retry, failure modes.\n- `references\u002Fslurm-preflight-storage.md` — extended preflight\u002Fstorage notes.\n- `references\u002Fslurm-execution-sdk.md` — extended execution\u002FSDK notes.\n- `references\u002Fdetailed-guide.md` — navigation map for the split references.\n",{"name":5,"tags":24,"license":15,"metadata":27,"description":7,"allowed-tools":30,"compatibility":31},[25,26],"platform","slurm",{"author":28,"version":29},"NVIDIA Corporation","0.1.0","Read Bash","Requires SSH access to a SLURM login node (passwordless via key auth) and SLURM_USER + SLURM_HOSTNAME env vars. The TAO SDK with the slurm extra (pip install 'nvidia-tao-sdk[slurm]') is needed only if you want Job handles, S3 I\u002FO wrapping, or run-folder durability via ActionWorkflow.","main","86cc27ffd16af6cb85948f1155578e6f3f61cc4c","skillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 11:52:53",{"source":8,"skillName":5,"commands":41,"prompt":51,"zipAvailable":52},[42,45,48],{"tool":43,"command":44},"npx","npx skills add Solizardking\u002Ftrain2earn --skill tao-run-on-slurm",{"tool":46,"command":47},"bunx","bunx skills add Solizardking\u002Ftrain2earn --skill tao-run-on-slurm",{"tool":49,"command":50},"pnpm","pnpm dlx skills add Solizardking\u002Ftrain2earn --skill tao-run-on-slurm","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002FSKILL.md",true,[54,60,64,68,72,76,80,84,88,92],{"path":55,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":58,"sourceHtmlUrl":59},"BENCHMARK.md","REFERENCE","参考文档",3950,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002FBENCHMARK.md",{"path":61,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":62,"sourceHtmlUrl":63},"metadata.json",529,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Fmetadata.json",{"path":65,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":66,"sourceHtmlUrl":67},"references\u002Fdetailed-guide.md",591,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Freferences\u002Fdetailed-guide.md",{"path":69,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":70,"sourceHtmlUrl":71},"references\u002Fskill_info.yaml",1282,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Freferences\u002Fskill_info.yaml",{"path":73,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":74,"sourceHtmlUrl":75},"references\u002Fslurm-container-execution.md",10110,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Freferences\u002Fslurm-container-execution.md",{"path":77,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":78,"sourceHtmlUrl":79},"references\u002Fslurm-execution-sdk.md",12926,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Freferences\u002Fslurm-execution-sdk.md",{"path":81,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fslurm-preflight-storage.md",11488,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Freferences\u002Fslurm-preflight-storage.md",{"path":85,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":86,"sourceHtmlUrl":87},"references\u002Fslurm-ssh-credentials.md",10389,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Freferences\u002Fslurm-ssh-credentials.md",{"path":89,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":90,"sourceHtmlUrl":91},"skill-card.md",3924,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Fskill-card.md",{"path":93,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":94,"sourceHtmlUrl":95},"verification.json",2567,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fpublic\u002Fapi\u002Fskills\u002Fnvidia\u002Ftao-run-on-slurm\u002Fverification.json",[97,105,113,121,128,136],{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":101,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":19},"01M78B94RSCAKMH1QS79VE6DG9","testing","For testing with Light Protocol programs and clients on localnet, devnet, and mainnet validation.","MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Ftesting\u002FSKILL.md",[],[],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":101,"licenseFileUrl":109,"sourceHtmlUrl":110,"categories":111,"keywords":112,"publishedAt":19},"01M78B81BKY8RX12HETPG8YKWK","qedgen","Formally verify programs by writing Lean 4 proofs. Trigger this skill whenever the user wants to formally verify code, generate Lean 4 proofs, prove properties about algorithms or smart contracts, verify invariants, convert program logic into formal specifications, or anything involving Lean 4 and formal verification. Also trigger when the user mentions \"qedgen\", \"lean proof\", \"formal proof\", \"verify my code\", \"prove correctness\", \"formal verification\", or wants mathematical guarantees about their implementation.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-formal-verification\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-formal-verification\u002FSKILL.md",[],[],{"skillNo":114,"name":115,"nameZh":6,"description":116,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":101,"licenseFileUrl":117,"sourceHtmlUrl":118,"categories":119,"keywords":120,"publishedAt":19},"01M78B7Q7C5K54E01DN83A60KG","solana-dev","Use when user asks to \"build a Solana dapp\", \"write an Anchor program\", \"create a token\", \"debug Solana errors\", \"set up wallet connection\", \"test my Solana program\", \"deploy to devnet\", or \"explain Solana concepts\" (rent, accounts, PDAs, CPIs, etc.). Also use for quick on-chain lookups via public RPC + curl — \"what's the balance of \u003Cwallet>\", \"look up transaction \u003Csig>\", \"token balance for \u003Caccount>\", \"check this address on mainnet\u002Fdevnet\". End-to-end Solana development playbook covering wallet connection, Anchor\u002FPinocchio programs, Codama client generation, LiteSVM\u002FMollusk\u002FSurfpool testing, security checklists, and JSON-RPC curl lookups against public clusters. Integrates with the Solana MCP server for live documentation search. Prefers framework-kit (@solana\u002Fclient + @solana\u002Freact-hooks) for UI, wallet-standard-first connection (incl. ConnectorKit), @solana\u002Fkit for client\u002FRPC code, and @solana\u002Fweb3-compat for legacy boundaries.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-dev\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-dev\u002FSKILL.md",[],[],{"skillNo":122,"name":123,"nameZh":6,"description":124,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":101,"licenseFileUrl":6,"sourceHtmlUrl":125,"categories":126,"keywords":127,"publishedAt":19},"01M78B7HXD6N6GWKSVRQ8F6Q31","solana-common-errors","Diagnose and fix common Solana development errors — GLIBC mismatches, Anchor version conflicts, cargo build-sbf \u002F platform-tools failures, LiteSVM issues, edition2024 crate pins, RPC airdrops, and Anchor 0.29→0.32 migrations. Use when builds fail, Anchor CLI won't install, or CI breaks on Debian\u002FUbuntu.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-common-errors\u002FSKILL.md",[],[],{"skillNo":129,"name":130,"nameZh":6,"description":131,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":132,"sourceHtmlUrl":133,"categories":134,"keywords":135,"publishedAt":19},"01M78B6T7YSYWQ0N4DP40QDXEZ","skill-creator","Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fskill-creator\u002Flicense.txt","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fskill-creator\u002FSKILL.md",[],[],{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":101,"licenseFileUrl":6,"sourceHtmlUrl":140,"categories":141,"keywords":142,"publishedAt":19},"01M78B1VFT9X5AND7AE1HZPZGF","phantom-wallet-mcp","Execute wallet operations through the Phantom MCP server. Use when the user wants to interact with their Phantom wallet directly — get addresses, transfer SOL or SPL tokens, buy\u002Fswap tokens, sign transactions, and sign messages across Solana, Ethereum, Bitcoin, and Sui. Requires the @phantom\u002Fmcp-server to be configured as an MCP server.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fphantom-wallet-mcp\u002FSKILL.md",[],[]]