cli-and-studio
Guides DataChain CLI commands, dataset and storage command routing, Studio auth/jobs/pipelines, and environment configuration.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- cli-and-studio
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Guides DataChain CLI commands, dataset and storage command routing, Studio auth/jobs/pipelines, and environment configuration.
- disable-model-invocation
- true
02 资源文件 6
- 参考文档references/cli-reference.md10.7 KB
- 参考文档references/configuration.md5.8 KB
- 参考文档references/studio-and-jobs.md10.4 KB
- 参考文档references/troubleshooting.md8.6 KB
- 脚本scripts/cli_help_smoke.py8.3 KB
- 脚本scripts/skill_layout_check.py4.9 KB
03 SKILL.md 正文
CLI and Studio
Use this sub-skill when the user asks how to choose or run datachain CLI
commands, diagnose command parser behavior, route storage versus dataset
operations, authenticate with DataChain Studio, run or monitor Studio jobs,
manage Studio pipelines, configure CLI/Studio environment variables, or inspect
DataChain bundled skill install targets.
Trigger Phrases
Load this sub-skill for prompts containing or implying:
datachain --help,datachain -V, verbose/quiet behavior, parser errors, or command-specific--helpoutput;- storage commands:
bucket status,cp,clone,ls,du,find,index, public/private buckets,--anon,--studio,--local, or--all; - dataset/display commands:
dataset,ds,dataset ls,dataset rm,dataset remove,dataset edit,dataset pull, orshow; - maintenance commands:
gc,clear-cache, outdated checkpoints, temporary tables, or local cache cleanup; - Studio commands:
auth login,auth token,auth team,auth logout,job run,job logs,job cancel,job clusters,pipeline create,pipeline list,pipeline pause,pipeline resume,pipeline status, orpipeline remove-job; datachain skill install,datachain skill list,--target,--local, Claude/Cursor/Codex/Pi/Copilot target selection, and safe layout previews;DATACHAIN_ROOT_DIR,DATACHAIN_GLOBAL_CONFIG_DIR,DATACHAIN_SYSTEM_CONFIG_DIR,DATACHAIN_NO_ANALYTICS,DATACHAIN_STUDIO_URL,DATACHAIN_STUDIO_TOKEN,DATACHAIN_STUDIO_TEAM,DATACHAIN_NAMESPACE,DATACHAIN_PROJECT, orDATACHAIN_IGNORE_CHECKPOINTS.
First Decision
- Need a command catalog or parser default → read
CLI reference. If the user only needs to
verify help output safely, run
cli_help_smoke.py with a command path such as
job runordataset ls; the helper appends--helpand avoids credentialed operations. - Need Studio auth, jobs, scheduling, logs, clusters, or pipelines → read Studio and jobs. Confirm token/team and scheduling intent before running any Studio mutation.
- Need environment variables, config precedence, namespace/project defaults, or token handling → read configuration.
- Need error diagnosis → read
troubleshooting, especially for missing
auth, local-vs-Studio flavor confusion, storage access,
show/schema confusion, job-log cancellation misconceptions, scheduled jobs, and parser errors. - Need DataChain bundled skill target layouts → use
skill_layout_check.py for a dry-run layout
preview, then route detailed target-layout and installed-skill behavior to
sibling
agent-harness.
Command Routing Rules
- Use storage commands when the subject is a URI/path or object-storage
listing/copy/index task (
s3://,gs://,az://,hf://,file://, local paths). Storagelsis not the same asdataset ls. - Use dataset commands when the subject is a named saved DataChain dataset
and version (
name,namespace.project.name, orname@version). Useshowto inspect dataset rows, schema, or query script. - Use Studio flavor flags deliberately. For
lsanddataset ls, no flavor flag defaults to local.--studiorequires Studio auth;--alllists both only when a token is available and otherwise falls back to local. - Use Studio job commands for remote execution of a query file.
job runsubmits work to Studio and may keep running after local log output stops. - Use pipeline commands for Studio-managed dependency pipelines over dataset updates. Pipelines are separate from Python SDK pipeline authoring.
- Use skill commands only for DataChain's bundled agent-skill installer; the
knowledge-base methodology and installed agent skill content are owned by
agent-harness.
Safety Rules
- Prefer
datachain <command> --helpor the bundled help smoke helper before composing unfamiliar commands. - Treat these as mutating:
cp,clone,index,dataset pull,dataset edit,dataset rm/remove,gc,clear-cache,auth login/logout/team,job run,job cancel,pipeline create/pause/resume/remove-job, andskill install/uninstall. - Before a mutating command, confirm the dataset name/version, output directory, Studio team, target agent, and whether the operation is local or Studio-scoped.
- Do not print, paste, commit, or store real Studio tokens, cloud credentials, or
secret
--envvalues.datachain auth tokenintentionally prints a secret; use it only when the user explicitly asks to inspect or copy the token. - For public buckets,
--anonis acceptable. For private storage, verify the credential source and considerbucket statusbefore listing or copying. - Closing
job runorjob logsoutput does not cancel a Studio job. Usedatachain job cancel <id>for cancellation. - Do not route Python SDK chain construction, UDF signatures,
.save(), orread_storagecode patterns here; send those to../sdk-pipelines/. - Do not route query expression operators/functions here; send those to
../query-engine/. - Do not route repo contributor testing, backend matrix development, or release
work here; send those to
../repo-development/.
Native Verification Candidates Owned Here
- Parser and flavor defaults: CLI parsing,
-v/-q,bucket status, anddetermine_flavorsunit coverage. - Skill command behavior: bundled skill list/install target parsing and layout rules; use the layout helper instead of performing real installs during usability checks.
- CLI end-to-end candidates: storage
ls/du/find/cp/clone, datasetls/edit/rm, andgc; mark network/object-copying steps as side-effecting unless run in a controlled fixture. - Studio candidates: auth login/logout/token/team, dataset Studio edit/remove, job run/log/cancel/clusters, reconnect behavior, checkpoint reuse, and scheduled-job behavior with mocked Studio responses.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).