[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78APPJ8W1H2QSB4Q8GDFMP6":3,"skill-same:01M78APPJ8W1H2QSB4Q8GDFMP6":95},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":33,"commitSha":34,"skillDirectory":35,"forksCount":14,"repositoryHtmlUrl":36,"repositoryHomepageUrl":6,"ownerHtmlUrl":37,"ownerBlogUrl":38,"ownerDescription":39,"updatedAt":40,"install":41,"resources":54},"01M78APPJ8W1H2QSB4Q8GDFMP6","tao-run-platform",null,"TAO Execution SDK for submitting and monitoring GPU training jobs on supported platforms (Brev, SLURM, local Docker, Kubernetes). Use when the user wants to run TAO jobs through the SDK, get job tracking, S3 I\u002FO wrapping, multi-node distributed training, or platform-specific features that docker-run can't provide. Trigger phrases include \"use the TAO SDK\", \"call tao_sdk\", \"AutoMLRunner\", \"ActionWorkflow\", \"Job handles\", \"S3 I\u002FO wrapping\", \"TAO platform run\".","Solizardking\u002Ftrain2earn","Solizardking","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F108096810?v=4","01M786KDB10AYV042N0F9BD5B5","Clawd","01M786KDC5C3NPE4W44C5641EZ",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: tao-run-platform\ndescription: TAO Execution SDK for submitting and monitoring GPU training jobs on supported platforms (Brev, SLURM,\n  local Docker, Kubernetes). Use when the user wants to run TAO jobs through the SDK, get job tracking, S3 I\u002FO wrapping,\n  multi-node distributed training, or platform-specific features that docker-run can't provide. Trigger phrases include\n  \"use the TAO SDK\", \"call tao_sdk\", \"AutoMLRunner\", \"ActionWorkflow\", \"Job handles\", \"S3 I\u002FO wrapping\", \"TAO platform run\".\nlicense: Apache-2.0\ncompatibility: Requires Python 3.10+ and the nvidia-tao-sdk package (pip install nvidia-tao-sdk[all]).\nmetadata:\n  author: NVIDIA Corporation\n  version: \"0.1.0\"\nallowed-tools: Read Bash\ntags:\n- platform\n- tao\n- sdk\n---\n\n# TAO Execution SDK\n\nThe SDK is the **optional** Python layer for users who need job handles, S3 I\u002FO wrapping, or platform-specific features (SLURM\u002FLustre queues, Kubernetes Jobs, local Docker debugging, Brev instance reuse). Most TAO skills run with just `docker run` and don't need it. Reach for the SDK when:\n\n- You want a `Job` handle to poll status and stream logs over time.\n- You need S3-aware input download \u002F output upload baked into the entrypoint.\n- You're chaining multiple jobs and want persisted state.\n\n## Preflight\n\nInstall `nvidia-tao-sdk[all]` before using this platform — the `[all]` extra pulls in every platform-specific dependency (Brev, S3 utilities, etc.). If it is missing, install it by default in the active Python environment and rerun the import check:\n\n```bash\npython -c \"import tao_sdk\" 2>\u002Fdev\u002Fnull || {\n  echo \"Installing missing Python requirement: nvidia-tao-sdk[all]\"\n  python -m pip install \"nvidia-tao-sdk[all]\"\n}\npython -c \"import tao_sdk\"\n```\n\nThe package index is environment-specific — the runner\u002Fcontainer is expected to have a working `pip` configuration (e.g. `~\u002F.pip\u002Fpip.conf`, `PIP_INDEX_URL`, `PIP_EXTRA_INDEX_URL`, or proxy). If the install fails for index\u002Fnetwork reasons, that's a runner setup issue; this skill stays agnostic to the registry.\n\nMissing pip requirements are installed automatically by default and reported in the run log. Non-pip\u002Fsystem prerequisites still require a normal preflight failure and user-visible remediation.\n\n## Setup\n\nCredentials come from **environment variables** — read from the session environment (export them in your shell before launching).\n\n```python\nfrom tao_sdk.platforms.brev   import BrevSDK     # Brev GPU instances\n\nsdk = BrevSDK()      # reads BREV_API_TOKEN (optional — falls back to brev login)\n```\n\nThe SDK validates credentials lazily on first use and raises `CredentialError` with a clear message if a required env var is missing. Required env vars:\n\n| Platform | Required | Optional |\n|---|---|---|\n| Brev | — (manual `brev login` works) | `BREV_API_TOKEN` |\n| S3 I\u002FO (any platform) | `S3_BUCKET_NAME`, `ACCESS_KEY`, `SECRET_KEY` | `S3_ENDPOINT_URL`, `CLOUD_REGION` |\n| Container env | `NGC_KEY` | `HF_TOKEN` |\n\nThe agent never reads credential values — it only checks presence with `[ -n \"$VAR_NAME\" ]`.\n\n## Workflow Launch Intake\n\nFor any TAO workflow or action launch, first confirm the user goal. Then ask\nfor platform and monitoring preferences before credentials or launch details.\nGenerate the supported platform choices from the packaged helper, not by\nscanning platform docs or folders:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Flist_tao_platforms.py \\\n  --skill-bank ${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external} --format text\n```\n\nAsk:\n\n1. Which supported platform should run this workflow?\n2. Should long-running monitoring stay enabled? Default: enabled. This means\n   the agent remains attached and posts status until terminal state, including\n   long `PENDING` queue waits.\n3. How many minutes between status updates? Default: 5 minutes.\n\nAfter the model\u002Faction are known, resolve the default container image from the\npackaged metadata and ask the user to confirm it or provide `image=\u003Coverride>`\nbefore creating runner files:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Fresolve_tao_image.py \\\n  --skill-bank ${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external} \\\n  --model \u003Cnetwork_arch> --action \u003Caction> --format text\n```\n\nFor train-capable model workflows, inspect model-level AutoML metadata before\ncreating a plain training job:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Flist_tao_models.py \\\n  --skill-bank ${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external} \\\n  --scope automl --format json\n```\n\nIf the selected model has `automl_enabled: true` and a valid train schema,\nroute training through `skills\u002Fapplications\u002Ftao-run-automl` by default with\n`automl_policy: on`. A workflow should only bypass AutoML when its run settings\ninclude `automl_policy: off`, the user explicitly asks for a plain run, or the\nmodel metadata says AutoML is enabled but the train schema is not packaged yet.\n\nAfter the platform is selected, get the credential filter:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Flist_tao_platforms.py \\\n  --skill-bank ${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external} \\\n  --platform \u003Cplatform> --format text\n```\n\nAsk only for credentials returned for the selected platform. For example, SLURM\nneeds `SLURM_USER` and `SLURM_HOSTNAME`; it does not need Brev credentials.\nKubernetes and local Docker do not need Brev or SLURM credentials. Ask storage\ncredentials such as S3 keys only when the selected platform and the data\u002Fresult\nURIs require them.\n\n## Core API\n\nAll platform SDKs implement the same core shape:\n\n```python\nsdk.create_job(image, command, gpu_count=1, env_vars=None, inputs=None, outputs=None, **kwargs) -> Job\nsdk.get_job_status(job_id) -> JobStatus\nsdk.get_job_logs(job_id, tail=None) -> str\nsdk.cancel_job(job_id) -> bool\nsdk.get_failure_analysis(job_id) -> dict | None\nsdk.get_job_results_dir(job_id) -> str\nsdk.check_path(remote_path) -> bool\nsdk.list_path(remote_path) -> list[str]\n```\n\nBrev-only:\n- `sdk.delete_instance(instance_id)` — clean up an ephemeral instance.\n- `sdk.list_instances()` — list active instances.\n\n## Submitting a Job\n\nThe agent always **constructs the container command via `build_entrypoint`** before calling `create_job`. The agent reads the action's schema from `skill_info.yaml` (`command`, `mode`, `config_format`, `inputs`, `outputs`, `upload_excludes`) and passes those fields as kwargs. `build_entrypoint` then bakes:\n\n1. The in-container `script_runner` runtime (inlined as a base64 heredoc — no need for `tao_sdk` to be installed in the container).\n2. The CLI invocation that, at runtime in the container, will: download declared inputs (S3 \u002F HF-Hub \u002F NGC), write the spec file at `{config_path}` with remote URIs rewritten to local paths, run the user command, and upload outputs.\n\nOutput destinations are resolved at runtime from env vars the SDK injects (see \"Where outputs go\" below). The platform SDK's `create_job` runs the resulting command **as-is** — no inputs\u002Foutputs kwargs, no implicit wrapping. The data flow is visible in the agent's code.\n\n### Where outputs go (resolved at runtime — agents don't manage it)\n\nThe SDK injects `TAO_JOB_ID` (matches `Job.id`) and, when a persistent mount is attached, `TAO_RESULTS_ROOT` into the container env. Inside the container, `script_runner` resolves output destinations:\n\n| Container env | Result |\n|---|---|\n| `TAO_RESULTS_ROOT` set (Lustre \u002F PVC \u002F bind \u002F NFS) | Outputs at `{TAO_RESULTS_ROOT}\u002F\u003Cjob_id>\u002F\u003Ckey>\u002F`; no upload |\n| `S3_BUCKET_NAME` set (cloud, no mount) | Outputs at `s3:\u002F\u002F{bucket}\u002Fresults\u002F\u003Cjob_id>\u002F\u003Ckey>\u002F`; uploaded at end of run |\n| Neither | Outputs at `\u002Fresults\u002F\u003Cjob_id>\u002F\u003Ckey>\u002F` (container-ephemeral) with a loud end-of-run warning |\n\nPer-platform policy:\n\n| SDK | What gets injected |\n|---|---|\n| `SlurmSDK` | `TAO_RESULTS_ROOT={SLURM_BASE_RESULTS_DIR}\u002Fresults` (always — Lustre, never S3, avoids GPU-idle scheduler kill) |\n| `KubernetesSDK` \u002F `DockerSDK` \u002F `BrevSDK` | `TAO_RESULTS_ROOT=\u002Fresults` if a mount targets `\u002Fresults`; otherwise S3 fallback |\n\nAgents who want a custom destination can put an `s3:\u002F\u002F...` URI or absolute path directly at the output spec key — explicit values override the auto-fill. Otherwise, model-natural defaults like cosmos-rl's `output_dir: \"output\"` or DINO's empty `results_dir` are auto-rewritten by `script_runner`.\n\n### The spec is nested dicts, NOT flat dotted keys\n\nThis is the most common mistake when constructing a spec. The dotted notation that appears in `skill_info.yaml`'s `inputs:` \u002F `outputs:` blocks (e.g. `section.subsection.key`) is a **path into** a nested spec — `script_runner` looks values up at that path. It's not the spec's own shape. The spec mirrors whatever shape the model's container expects (typically a nested TOML\u002FYAML).\n\n```python\n# ✓ CORRECT — nested dicts\nspecs = {\n    \"section\": {\n        \"subsection\": {\"key\": \"value\"},\n    },\n}\n\n# ✗ WRONG — flat top-level key with dots. TOML\u002FYAML emits this as a\n# quoted bare-string key, the model sees an empty `section` table, and\n# any input declared at \"section.subsection.key\" silently fails to\n# download because _get_nested(specs, \"section.subsection.key\") → None.\nspecs = {\n    \"section.subsection.key\": \"value\",\n}\n```\n\nThe two shapes look superficially similar but mean different things. When in doubt, open the model's `references\u002F` directory (e.g. a default-spec TOML or YAML) — that's the literal nested structure the spec dict needs to mirror. The `inputs:` \u002F `outputs:` declarations in `skill_info.yaml` are *paths into* the nested spec, not key names.\n\n### Constructing the spec \u002F args\n\nThe skill's action declares its config mechanism in `skill_info.yaml`'s `actions.\u003Caction>.mode` field. Treat missing `mode` as invalid metadata and fix the skill instead of inferring a default. Read `actions.\u003Caction>.mode` first, then pass the matching argument shape to `build_entrypoint`:\n\n| Declared mode | What the agent passes |\n|---|---|\n| `config` | `specs=...` with spec-keyed `inputs` \u002F `outputs`; the helper writes the spec file, rewrites URIs, and runs the command |\n| `args` | `args=...` with optional spec-keyed `inputs` \u002F `outputs`; the helper substitutes CLI args into the command template |\n| `passthrough` | path-keyed `inputs=...` and\u002For `outputs=...`; the helper downloads to listed paths, runs the command, and uploads listed outputs |\n\nDo not infer mode from missing metadata. Missing `mode` means the skill contract is stale.\n\nSee [`references\u002Fspec-construction.md`](references\u002Fspec-construction.md) for the per-mode construction strategy, the recommended decision order, and worked `build_entrypoint` examples for spec-driven jobs (config file) and path-keyed jobs (no config file).\n\n## Resolving container images\n\nSkills declare images either by key (`tao_toolkit.pyt`) or as an absolute URI (`nvcr.io\u002F...`). Use `resolve_container_image()` to handle both:\n\n```python\nfrom tao_sdk.versions import resolve_container_image\nimage = resolve_container_image(skill_info[\"container_image\"])\n```\n\nBehind the scenes it walks `versions.yaml` for keys; absolute URIs are returned as-is.\n\n## Monitoring\n\n```python\nstatus = sdk.get_job_status(job.id)\nprint(status.status)   # Pending, Running, Complete, Error, Canceled\nprint(status.message)  # platform-specific detail\n\nlogs = sdk.get_job_logs(job.id, tail=200)\nprint(logs)\n```\n\nOn failure, `get_failure_analysis()` classifies the root cause:\n\n```python\nanalysis = sdk.get_failure_analysis(job.id)\nif analysis:\n    print(analysis[\"err_class\"])   # ERR_PROGRAM, ERR_INFRA, etc.\n    print(analysis[\"suggestion\"])  # human-readable fix\n    for event in analysis.get(\"job_failure_by_node_event\", []):\n        print(event[\"node_event_name\"], event[\"message\"])  # OOM, GPU error, etc.\n```\n\n## Polling pattern\n\nFor interactive runs where the user wants to watch:\n\n```python\nimport time\nstatus_interval_minutes = status_interval_minutes or 5\nwhile True:\n    status = sdk.get_job_status(job.id)\n    if status.status in (\"Complete\", \"Error\", \"Canceled\"):\n        break\n    print(f\"  {status.status}\")\n    time.sleep(status_interval_minutes * 60)\n\nif status.status == \"Error\":\n    print(sdk.get_job_logs(job.id, tail=100))\n    print(sdk.get_failure_analysis(job.id))\n```\n\nWith long-running monitoring enabled, do not stop after 30 minutes or after a\nfew unchanged polls. Keep emitting updates every `status_interval_minutes`\nuntil the job finishes, fails, is canceled, or the user asks to detach\u002Fstop.\nIf the chat\u002Fruntime cannot remain open that long, say so explicitly and provide\nthe durable workflow\u002Flog path for manual status refresh.\n\nDo not use a final response for non-terminal monitored jobs. Finalizing the\nturn detaches the chat watcher. Keep non-terminal status messages in progress\nupdates and continue polling; only finalize at terminal state, explicit user\ndetach\u002Fstop, or a real runtime limit that prevents further polling.\n\nFor background runs, persist `job.id` and the `state_file` path, then re-attach later by constructing the same SDK and calling `get_job_status(job_id)` — job state is read from the on-disk store.\n\n## Orchestration patterns\n\nMulti-step workflows, parallel sweeps, and run-folder durability via\n`ActionWorkflow` live in\n[`references\u002Forchestration-patterns.md`](references\u002Forchestration-patterns.md).\nRead it before chaining `create_job` calls, sweeping a parameter, or\npersisting run state across context breaks.\n\n## Dataset utilities\n\nWhen the skill's documented filenames don't match the user's layout, list the dataset to confirm:\n\n```python\nassert sdk.check_path(\"s3:\u002F\u002Fmy-bucket\u002Fcoco\u002F\")\nfiles = sdk.list_path(\"s3:\u002F\u002Fmy-bucket\u002Fcoco\u002Ftrain\u002F\")\n# Use the actual paths to set spec fields.\n```\n\nFor S3 paths, strip trailing slashes when concatenating to avoid `\u002F\u002F`:\n\n```python\nbase = dataset_uri.rstrip(\"\u002F\")\nspecs[\"dataset\"][\"train_csv\"] = f\"{base}\u002Ftrain.csv\"   # nested — see \"spec is nested dicts\"\n```\n\n## Platform-specific notes\n\nSee [`references\u002Fplatform-notes.md`](references\u002Fplatform-notes.md) for per-platform behavior, kwargs, and credential scoping: Brev (`instance_id`\u002F`gpu_type`\u002F`cloud_cred_id`\u002F`workspace_group_id`, ready-wait timeouts), SLURM (sbatch over SSH, Lustre paths, queue defaults), Kubernetes (kubeconfig, GPU Operator), and local Docker (single-host, multi-GPU).\n\n## Error patterns\n\nSDK error → root cause → fix mappings are in\n[`references\u002Ferror-patterns.md`](references\u002Ferror-patterns.md). Read when\nyou hit a `CredentialError`, image-pull failure, stuck-Pending job, or\nsimilar — the entries map exception text to the underlying cause.\n\n## What the SDK does NOT do\n\nThe SDK does not read\u002Finterpret skills, run AutoML on its own, decide spec contents, select platforms, or orchestrate multi-step workflows — those stay the agent's responsibility. See [`references\u002Fscope.md`](references\u002Fscope.md) for the full scope guardrails, including the model-level AutoML policy (`automl_enabled: true` → `skills\u002Fapplications\u002Ftao-run-automl` unless `automl_policy: off` or the user asks for a plain single run).\n","# TAO Execution SDK\n\nThe SDK is the **optional** Python layer for users who need job handles, S3 I\u002FO wrapping, or platform-specific features (SLURM\u002FLustre queues, Kubernetes Jobs, local Docker debugging, Brev instance reuse). Most TAO skills run with just `docker run` and don't need it. Reach for the SDK when:\n\n- You want a `Job` handle to poll status and stream logs over time.\n- You need S3-aware input download \u002F output upload baked into the entrypoint.\n- You're chaining multiple jobs and want persisted state.\n\n## Preflight\n\nInstall `nvidia-tao-sdk[all]` before using this platform — the `[all]` extra pulls in every platform-specific dependency (Brev, S3 utilities, etc.). If it is missing, install it by default in the active Python environment and rerun the import check:\n\n```bash\npython -c \"import tao_sdk\" 2>\u002Fdev\u002Fnull || {\n  echo \"Installing missing Python requirement: nvidia-tao-sdk[all]\"\n  python -m pip install \"nvidia-tao-sdk[all]\"\n}\npython -c \"import tao_sdk\"\n```\n\nThe package index is environment-specific — the runner\u002Fcontainer is expected to have a working `pip` configuration (e.g. `~\u002F.pip\u002Fpip.conf`, `PIP_INDEX_URL`, `PIP_EXTRA_INDEX_URL`, or proxy). If the install fails for index\u002Fnetwork reasons, that's a runner setup issue; this skill stays agnostic to the registry.\n\nMissing pip requirements are installed automatically by default and reported in the run log. Non-pip\u002Fsystem prerequisites still require a normal preflight failure and user-visible remediation.\n\n## Setup\n\nCredentials come from **environment variables** — read from the session environment (export them in your shell before launching).\n\n```python\nfrom tao_sdk.platforms.brev   import BrevSDK     # Brev GPU instances\n\nsdk = BrevSDK()      # reads BREV_API_TOKEN (optional — falls back to brev login)\n```\n\nThe SDK validates credentials lazily on first use and raises `CredentialError` with a clear message if a required env var is missing. Required env vars:\n\n| Platform | Required | Optional |\n|---|---|---|\n| Brev | — (manual `brev login` works) | `BREV_API_TOKEN` |\n| S3 I\u002FO (any platform) | `S3_BUCKET_NAME`, `ACCESS_KEY`, `SECRET_KEY` | `S3_ENDPOINT_URL`, `CLOUD_REGION` |\n| Container env | `NGC_KEY` | `HF_TOKEN` |\n\nThe agent never reads credential values — it only checks presence with `[ -n \"$VAR_NAME\" ]`.\n\n## Workflow Launch Intake\n\nFor any TAO workflow or action launch, first confirm the user goal. Then ask\nfor platform and monitoring preferences before credentials or launch details.\nGenerate the supported platform choices from the packaged helper, not by\nscanning platform docs or folders:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Flist_tao_platforms.py \\\n  --skill-bank ${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external} --format text\n```\n\nAsk:\n\n1. Which supported platform should run this workflow?\n2. Should long-running monitoring stay enabled? Default: enabled. This means\n   the agent remains attached and posts status until terminal state, including\n   long `PENDING` queue waits.\n3. How many minutes between status updates? Default: 5 minutes.\n\nAfter the model\u002Faction are known, resolve the default container image from the\npackaged metadata and ask the user to confirm it or provide `image=\u003Coverride>`\nbefore creating runner files:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Fresolve_tao_image.py \\\n  --skill-bank ${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external} \\\n  --model \u003Cnetwork_arch> --action \u003Caction> --format text\n```\n\nFor train-capable model workflows, inspect model-level AutoML metadata before\ncreating a plain training job:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Flist_tao_models.py \\\n  --skill-bank ${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external} \\\n  --scope automl --format json\n```\n\nIf the selected model has `automl_enabled: true` and a valid train schema,\nroute training through `skills\u002Fapplications\u002Ftao-run-automl` by default with\n`automl_policy: on`. A workflow should only bypass AutoML when its run settings\ninclude `automl_policy: off`, the user explicitly asks for a plain run, or the\nmodel metadata says AutoML is enabled but the train schema is not packaged yet.\n\nAfter the platform is selected, get the credential filter:\n\n```bash\n${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external}\u002Fscripts\u002Flist_tao_platforms.py \\\n  --skill-bank ${TAO_SKILL_BANK_PATH:-~\u002Ftao-skills-external} \\\n  --platform \u003Cplatform> --format text\n```\n\nAsk only for credentials returned for the selected platform. For example, SLURM\nneeds `SLURM_USER` and `SLURM_HOSTNAME`; it does not need Brev credentials.\nKubernetes and local Docker do not need Brev or SLURM credentials. Ask storage\ncredentials such as S3 keys only when the selected platform and the data\u002Fresult\nURIs require them.\n\n## Core API\n\nAll platform SDKs implement the same core shape:\n\n```python\nsdk.create_job(image, command, gpu_count=1, env_vars=None, inputs=None, outputs=None, **kwargs) -> Job\nsdk.get_job_status(job_id) -> JobStatus\nsdk.get_job_logs(job_id, tail=None) -> str\nsdk.cancel_job(job_id) -> bool\nsdk.get_failure_analysis(job_id) -> dict | None\nsdk.get_job_results_dir(job_id) -> str\nsdk.check_path(remote_path) -> bool\nsdk.list_path(remote_path) -> list[str]\n```\n\nBrev-only:\n- `sdk.delete_instance(instance_id)` — clean up an ephemeral instance.\n- `sdk.list_instances()` — list active instances.\n\n## Submitting a Job\n\nThe agent always **constructs the container command via `build_entrypoint`** before calling `create_job`. The agent reads the action's schema from `skill_info.yaml` (`command`, `mode`, `config_format`, `inputs`, `outputs`, `upload_excludes`) and passes those fields as kwargs. `build_entrypoint` then bakes:\n\n1. The in-container `script_runner` runtime (inlined as a base64 heredoc — no need for `tao_sdk` to be installed in the container).\n2. The CLI invocation that, at runtime in the container, will: download declared inputs (S3 \u002F HF-Hub \u002F NGC), write the spec file at `{config_path}` with remote URIs rewritten to local paths, run the user command, and upload outputs.\n\nOutput destinations are resolved at runtime from env vars the SDK injects (see \"Where outputs go\" below). The platform SDK's `create_job` runs the resulting command **as-is** — no inputs\u002Foutputs kwargs, no implicit wrapping. The data flow is visible in the agent's code.\n\n### Where outputs go (resolved at runtime — agents don't manage it)\n\nThe SDK injects `TAO_JOB_ID` (matches `Job.id`) and, when a persistent mount is attached, `TAO_RESULTS_ROOT` into the container env. Inside the container, `script_runner` resolves output destinations:\n\n| Container env | Result |\n|---|---|\n| `TAO_RESULTS_ROOT` set (Lustre \u002F PVC \u002F bind \u002F NFS) | Outputs at `{TAO_RESULTS_ROOT}\u002F\u003Cjob_id>\u002F\u003Ckey>\u002F`; no upload |\n| `S3_BUCKET_NAME` set (cloud, no mount) | Outputs at `s3:\u002F\u002F{bucket}\u002Fresults\u002F\u003Cjob_id>\u002F\u003Ckey>\u002F`; uploaded at end of run |\n| Neither | Outputs at `\u002Fresults\u002F\u003Cjob_id>\u002F\u003Ckey>\u002F` (container-ephemeral) with a loud end-of-run warning |\n\nPer-platform policy:\n\n| SDK | What gets injected |\n|---|---|\n| `SlurmSDK` | `TAO_RESULTS_ROOT={SLURM_BASE_RESULTS_DIR}\u002Fresults` (always — Lustre, never S3, avoids GPU-idle scheduler kill) |\n| `KubernetesSDK` \u002F `DockerSDK` \u002F `BrevSDK` | `TAO_RESULTS_ROOT=\u002Fresults` if a mount targets `\u002Fresults`; otherwise S3 fallback |\n\nAgents who want a custom destination can put an `s3:\u002F\u002F...` URI or absolute path directly at the output spec key — explicit values override the auto-fill. Otherwise, model-natural defaults like cosmos-rl's `output_dir: \"output\"` or DINO's empty `results_dir` are auto-rewritten by `script_runner`.\n\n### The spec is nested dicts, NOT flat dotted keys\n\nThis is the most common mistake when constructing a spec. The dotted notation that appears in `skill_info.yaml`'s `inputs:` \u002F `outputs:` blocks (e.g. `section.subsection.key`) is a **path into** a nested spec — `script_runner` looks values up at that path. It's not the spec's own shape. The spec mirrors whatever shape the model's container expects (typically a nested TOML\u002FYAML).\n\n```python\n# ✓ CORRECT — nested dicts\nspecs = {\n    \"section\": {\n        \"subsection\": {\"key\": \"value\"},\n    },\n}\n\n# ✗ WRONG — flat top-level key with dots. TOML\u002FYAML emits this as a\n# quoted bare-string key, the model sees an empty `section` table, and\n# any input declared at \"section.subsection.key\" silently fails to\n# download because _get_nested(specs, \"section.subsection.key\") → None.\nspecs = {\n    \"section.subsection.key\": \"value\",\n}\n```\n\nThe two shapes look superficially similar but mean different things. When in doubt, open the model's `references\u002F` directory (e.g. a default-spec TOML or YAML) — that's the literal nested structure the spec dict needs to mirror. The `inputs:` \u002F `outputs:` declarations in `skill_info.yaml` are *paths into* the nested spec, not key names.\n\n### Constructing the spec \u002F args\n\nThe skill's action declares its config mechanism in `skill_info.yaml`'s `actions.\u003Caction>.mode` field. Treat missing `mode` as invalid metadata and fix the skill instead of inferring a default. Read `actions.\u003Caction>.mode` first, then pass the matching argument shape to `build_entrypoint`:\n\n| Declared mode | What the agent passes |\n|---|---|\n| `config` | `specs=...` with spec-keyed `inputs` \u002F `outputs`; the helper writes the spec file, rewrites URIs, and runs the command |\n| `args` | `args=...` with optional spec-keyed `inputs` \u002F `outputs`; the helper substitutes CLI args into the command template |\n| `passthrough` | path-keyed `inputs=...` and\u002For `outputs=...`; the helper downloads to listed paths, runs the command, and uploads listed outputs |\n\nDo not infer mode from missing metadata. Missing `mode` means the skill contract is stale.\n\nSee [`references\u002Fspec-construction.md`](references\u002Fspec-construction.md) for the per-mode construction strategy, the recommended decision order, and worked `build_entrypoint` examples for spec-driven jobs (config file) and path-keyed jobs (no config file).\n\n## Resolving container images\n\nSkills declare images either by key (`tao_toolkit.pyt`) or as an absolute URI (`nvcr.io\u002F...`). Use `resolve_container_image()` to handle both:\n\n```python\nfrom tao_sdk.versions import resolve_container_image\nimage = resolve_container_image(skill_info[\"container_image\"])\n```\n\nBehind the scenes it walks `versions.yaml` for keys; absolute URIs are returned as-is.\n\n## Monitoring\n\n```python\nstatus = sdk.get_job_status(job.id)\nprint(status.status)   # Pending, Running, Complete, Error, Canceled\nprint(status.message)  # platform-specific detail\n\nlogs = sdk.get_job_logs(job.id, tail=200)\nprint(logs)\n```\n\nOn failure, `get_failure_analysis()` classifies the root cause:\n\n```python\nanalysis = sdk.get_failure_analysis(job.id)\nif analysis:\n    print(analysis[\"err_class\"])   # ERR_PROGRAM, ERR_INFRA, etc.\n    print(analysis[\"suggestion\"])  # human-readable fix\n    for event in analysis.get(\"job_failure_by_node_event\", []):\n        print(event[\"node_event_name\"], event[\"message\"])  # OOM, GPU error, etc.\n```\n\n## Polling pattern\n\nFor interactive runs where the user wants to watch:\n\n```python\nimport time\nstatus_interval_minutes = status_interval_minutes or 5\nwhile True:\n    status = sdk.get_job_status(job.id)\n    if status.status in (\"Complete\", \"Error\", \"Canceled\"):\n        break\n    print(f\"  {status.status}\")\n    time.sleep(status_interval_minutes * 60)\n\nif status.status == \"Error\":\n    print(sdk.get_job_logs(job.id, tail=100))\n    print(sdk.get_failure_analysis(job.id))\n```\n\nWith long-running monitoring enabled, do not stop after 30 minutes or after a\nfew unchanged polls. Keep emitting updates every `status_interval_minutes`\nuntil the job finishes, fails, is canceled, or the user asks to detach\u002Fstop.\nIf the chat\u002Fruntime cannot remain open that long, say so explicitly and provide\nthe durable workflow\u002Flog path for manual status refresh.\n\nDo not use a final response for non-terminal monitored jobs. Finalizing the\nturn detaches the chat watcher. Keep non-terminal status messages in progress\nupdates and continue polling; only finalize at terminal state, explicit user\ndetach\u002Fstop, or a real runtime limit that prevents further polling.\n\nFor background runs, persist `job.id` and the `state_file` path, then re-attach later by constructing the same SDK and calling `get_job_status(job_id)` — job state is read from the on-disk store.\n\n## Orchestration patterns\n\nMulti-step workflows, parallel sweeps, and run-folder durability via\n`ActionWorkflow` live in\n[`references\u002Forchestration-patterns.md`](references\u002Forchestration-patterns.md).\nRead it before chaining `create_job` calls, sweeping a parameter, or\npersisting run state across context breaks.\n\n## Dataset utilities\n\nWhen the skill's documented filenames don't match the user's layout, list the dataset to confirm:\n\n```python\nassert sdk.check_path(\"s3:\u002F\u002Fmy-bucket\u002Fcoco\u002F\")\nfiles = sdk.list_path(\"s3:\u002F\u002Fmy-bucket\u002Fcoco\u002Ftrain\u002F\")\n# Use the actual paths to set spec fields.\n```\n\nFor S3 paths, strip trailing slashes when concatenating to avoid `\u002F\u002F`:\n\n```python\nbase = dataset_uri.rstrip(\"\u002F\")\nspecs[\"dataset\"][\"train_csv\"] = f\"{base}\u002Ftrain.csv\"   # nested — see \"spec is nested dicts\"\n```\n\n## Platform-specific notes\n\nSee [`references\u002Fplatform-notes.md`](references\u002Fplatform-notes.md) for per-platform behavior, kwargs, and credential scoping: Brev (`instance_id`\u002F`gpu_type`\u002F`cloud_cred_id`\u002F`workspace_group_id`, ready-wait timeouts), SLURM (sbatch over SSH, Lustre paths, queue defaults), Kubernetes (kubeconfig, GPU Operator), and local Docker (single-host, multi-GPU).\n\n## Error patterns\n\nSDK error → root cause → fix mappings are in\n[`references\u002Ferror-patterns.md`](references\u002Ferror-patterns.md). Read when\nyou hit a `CredentialError`, image-pull failure, stuck-Pending job, or\nsimilar — the entries map exception text to the underlying cause.\n\n## What the SDK does NOT do\n\nThe SDK does not read\u002Finterpret skills, run AutoML on its own, decide spec contents, select platforms, or orchestrate multi-step workflows — those stay the agent's responsibility. See [`references\u002Fscope.md`](references\u002Fscope.md) for the full scope guardrails, including the model-level AutoML policy (`automl_enabled: true` → `skills\u002Fapplications\u002Ftao-run-automl` unless `automl_policy: off` or the user asks for a plain single run).\n",{"name":5,"tags":24,"license":15,"metadata":28,"description":7,"allowed-tools":31,"compatibility":32},[25,26,27],"platform","tao","sdk",{"author":29,"version":30},"NVIDIA Corporation","0.1.0","Read Bash","Requires Python 3.10+ and the nvidia-tao-sdk package (pip install nvidia-tao-sdk[all]).","main","86cc27ffd16af6cb85948f1155578e6f3f61cc4c","skillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn","https:\u002F\u002Fgithub.com\u002FSolizardking","https:\u002F\u002Fmusebook.trade","Got a mile high and never came down","2026-10-08 11:52:53",{"source":8,"skillName":5,"commands":42,"prompt":52,"zipAvailable":53},[43,46,49],{"tool":44,"command":45},"npx","npx skills add Solizardking\u002Ftrain2earn --skill tao-run-platform",{"tool":47,"command":48},"bunx","bunx skills add Solizardking\u002Ftrain2earn --skill tao-run-platform",{"tool":50,"command":51},"pnpm","pnpm dlx skills add Solizardking\u002Ftrain2earn --skill tao-run-platform","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002FSKILL.md",true,[55,61,65,69,73,77,81,85,89],{"path":56,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":59,"sourceHtmlUrl":60},"BENCHMARK.md","REFERENCE","参考文档",3950,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002FBENCHMARK.md",{"path":62,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":63,"sourceHtmlUrl":64},"evals\u002Fevals.json",759,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002Fevals\u002Fevals.json",{"path":66,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":67,"sourceHtmlUrl":68},"references\u002Ferror-patterns.md",1337,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002Freferences\u002Ferror-patterns.md",{"path":70,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":71,"sourceHtmlUrl":72},"references\u002Forchestration-patterns.md",2655,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002Freferences\u002Forchestration-patterns.md",{"path":74,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":75,"sourceHtmlUrl":76},"references\u002Fplatform-notes.md",2468,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002Freferences\u002Fplatform-notes.md",{"path":78,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":79,"sourceHtmlUrl":80},"references\u002Fscope.md",1287,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002Freferences\u002Fscope.md",{"path":82,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":83,"sourceHtmlUrl":84},"references\u002Fspec-construction.md",7017,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002Freferences\u002Fspec-construction.md",{"path":86,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":87,"sourceHtmlUrl":88},"skill-card.md",3959,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002Fskill-card.md",{"path":90,"resourceType":91,"resourceTypeDesc":92,"sizeBytes":93,"sourceHtmlUrl":94},"skill.oms.sig","OTHER","其他",5793,"https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fnvidia\u002Ftao-run-platform\u002Fskill.oms.sig",[96,104,112,120,127,135],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":100,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":103,"publishedAt":19},"01M78B94RSCAKMH1QS79VE6DG9","testing","For testing with Light Protocol programs and clients on localnet, devnet, and mainnet validation.","MIT","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Ftesting\u002FSKILL.md",[],[],{"skillNo":105,"name":106,"nameZh":6,"description":107,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":100,"licenseFileUrl":108,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":19},"01M78B81BKY8RX12HETPG8YKWK","qedgen","Formally verify programs by writing Lean 4 proofs. Trigger this skill whenever the user wants to formally verify code, generate Lean 4 proofs, prove properties about algorithms or smart contracts, verify invariants, convert program logic into formal specifications, or anything involving Lean 4 and formal verification. Also trigger when the user mentions \"qedgen\", \"lean proof\", \"formal proof\", \"verify my code\", \"prove correctness\", \"formal verification\", or wants mathematical guarantees about their implementation.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-formal-verification\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-formal-verification\u002FSKILL.md",[],[],{"skillNo":113,"name":114,"nameZh":6,"description":115,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":100,"licenseFileUrl":116,"sourceHtmlUrl":117,"categories":118,"keywords":119,"publishedAt":19},"01M78B7Q7C5K54E01DN83A60KG","solana-dev","Use when user asks to \"build a Solana dapp\", \"write an Anchor program\", \"create a token\", \"debug Solana errors\", \"set up wallet connection\", \"test my Solana program\", \"deploy to devnet\", or \"explain Solana concepts\" (rent, accounts, PDAs, CPIs, etc.). Also use for quick on-chain lookups via public RPC + curl — \"what's the balance of \u003Cwallet>\", \"look up transaction \u003Csig>\", \"token balance for \u003Caccount>\", \"check this address on mainnet\u002Fdevnet\". End-to-end Solana development playbook covering wallet connection, Anchor\u002FPinocchio programs, Codama client generation, LiteSVM\u002FMollusk\u002FSurfpool testing, security checklists, and JSON-RPC curl lookups against public clusters. Integrates with the Solana MCP server for live documentation search. Prefers framework-kit (@solana\u002Fclient + @solana\u002Freact-hooks) for UI, wallet-standard-first connection (incl. ConnectorKit), @solana\u002Fkit for client\u002FRPC code, and @solana\u002Fweb3-compat for legacy boundaries.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-dev\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-dev\u002FSKILL.md",[],[],{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":100,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":126,"publishedAt":19},"01M78B7HXD6N6GWKSVRQ8F6Q31","solana-common-errors","Diagnose and fix common Solana development errors — GLIBC mismatches, Anchor version conflicts, cargo build-sbf \u002F platform-tools failures, LiteSVM issues, edition2024 crate pins, RPC airdrops, and Anchor 0.29→0.32 migrations. Use when builds fail, Anchor CLI won't install, or CI breaks on Debian\u002FUbuntu.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fsolana-common-errors\u002FSKILL.md",[],[],{"skillNo":128,"name":129,"nameZh":6,"description":130,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":131,"sourceHtmlUrl":132,"categories":133,"keywords":134,"publishedAt":19},"01M78B6T7YSYWQ0N4DP40QDXEZ","skill-creator","Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fskill-creator\u002Flicense.txt","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fskill-creator\u002FSKILL.md",[],[],{"skillNo":136,"name":137,"nameZh":6,"description":138,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":100,"licenseFileUrl":6,"sourceHtmlUrl":139,"categories":140,"keywords":141,"publishedAt":19},"01M78B1VFT9X5AND7AE1HZPZGF","phantom-wallet-mcp","Execute wallet operations through the Phantom MCP server. Use when the user wants to interact with their Phantom wallet directly — get addresses, transfer SOL or SPL tokens, buy\u002Fswap tokens, sign transactions, and sign messages across Solana, Ethereum, Bitcoin, and Sui. Requires the @phantom\u002Fmcp-server to be configured as an MCP server.","https:\u002F\u002Fgithub.com\u002FSolizardking\u002Ftrain2earn\u002Fblob\u002Fmain\u002Fskillhub\u002Fskills\u002Fphantom-wallet-mcp\u002FSKILL.md",[],[]]