AGENT SKILL

cli-and-studio

Guides DataChain CLI commands, dataset and storage command routing, Studio auth/jobs/pipelines, and environment configuration.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
cli-and-studio
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Guides DataChain CLI commands, dataset and storage command routing, Studio auth/jobs/pipelines, and environment configuration.
disable-model-invocation
true

02 资源文件 6

03 SKILL.md 正文

CLI and Studio

Use this sub-skill when the user asks how to choose or run datachain CLI commands, diagnose command parser behavior, route storage versus dataset operations, authenticate with DataChain Studio, run or monitor Studio jobs, manage Studio pipelines, configure CLI/Studio environment variables, or inspect DataChain bundled skill install targets.

Trigger Phrases

Load this sub-skill for prompts containing or implying:

  • datachain --help, datachain -V, verbose/quiet behavior, parser errors, or command-specific --help output;
  • storage commands: bucket status, cp, clone, ls, du, find, index, public/private buckets, --anon, --studio, --local, or --all;
  • dataset/display commands: dataset, ds, dataset ls, dataset rm, dataset remove, dataset edit, dataset pull, or show;
  • maintenance commands: gc, clear-cache, outdated checkpoints, temporary tables, or local cache cleanup;
  • Studio commands: auth login, auth token, auth team, auth logout, job run, job logs, job cancel, job clusters, pipeline create, pipeline list, pipeline pause, pipeline resume, pipeline status, or pipeline remove-job;
  • datachain skill install, datachain skill list, --target, --local, Claude/Cursor/Codex/Pi/Copilot target selection, and safe layout previews;
  • DATACHAIN_ROOT_DIR, DATACHAIN_GLOBAL_CONFIG_DIR, DATACHAIN_SYSTEM_CONFIG_DIR, DATACHAIN_NO_ANALYTICS, DATACHAIN_STUDIO_URL, DATACHAIN_STUDIO_TOKEN, DATACHAIN_STUDIO_TEAM, DATACHAIN_NAMESPACE, DATACHAIN_PROJECT, or DATACHAIN_IGNORE_CHECKPOINTS.

First Decision

  1. Need a command catalog or parser default → read CLI reference. If the user only needs to verify help output safely, run cli_help_smoke.py with a command path such as job run or dataset ls; the helper appends --help and avoids credentialed operations.
  2. Need Studio auth, jobs, scheduling, logs, clusters, or pipelines → read Studio and jobs. Confirm token/team and scheduling intent before running any Studio mutation.
  3. Need environment variables, config precedence, namespace/project defaults, or token handling → read configuration.
  4. Need error diagnosis → read troubleshooting, especially for missing auth, local-vs-Studio flavor confusion, storage access, show/schema confusion, job-log cancellation misconceptions, scheduled jobs, and parser errors.
  5. Need DataChain bundled skill target layouts → use skill_layout_check.py for a dry-run layout preview, then route detailed target-layout and installed-skill behavior to sibling agent-harness.

Command Routing Rules

  • Use storage commands when the subject is a URI/path or object-storage listing/copy/index task (s3://, gs://, az://, hf://, file://, local paths). Storage ls is not the same as dataset ls.
  • Use dataset commands when the subject is a named saved DataChain dataset and version (name, namespace.project.name, or name@version). Use show to inspect dataset rows, schema, or query script.
  • Use Studio flavor flags deliberately. For ls and dataset ls, no flavor flag defaults to local. --studio requires Studio auth; --all lists both only when a token is available and otherwise falls back to local.
  • Use Studio job commands for remote execution of a query file. job run submits work to Studio and may keep running after local log output stops.
  • Use pipeline commands for Studio-managed dependency pipelines over dataset updates. Pipelines are separate from Python SDK pipeline authoring.
  • Use skill commands only for DataChain's bundled agent-skill installer; the knowledge-base methodology and installed agent skill content are owned by agent-harness.

Safety Rules

  • Prefer datachain <command> --help or the bundled help smoke helper before composing unfamiliar commands.
  • Treat these as mutating: cp, clone, index, dataset pull, dataset edit, dataset rm/remove, gc, clear-cache, auth login/logout/team, job run, job cancel, pipeline create/pause/resume/remove-job, and skill install/uninstall.
  • Before a mutating command, confirm the dataset name/version, output directory, Studio team, target agent, and whether the operation is local or Studio-scoped.
  • Do not print, paste, commit, or store real Studio tokens, cloud credentials, or secret --env values. datachain auth token intentionally prints a secret; use it only when the user explicitly asks to inspect or copy the token.
  • For public buckets, --anon is acceptable. For private storage, verify the credential source and consider bucket status before listing or copying.
  • Closing job run or job logs output does not cancel a Studio job. Use datachain job cancel <id> for cancellation.
  • Do not route Python SDK chain construction, UDF signatures, .save(), or read_storage code patterns here; send those to ../sdk-pipelines/.
  • Do not route query expression operators/functions here; send those to ../query-engine/.
  • Do not route repo contributor testing, backend matrix development, or release work here; send those to ../repo-development/.

Native Verification Candidates Owned Here

  • Parser and flavor defaults: CLI parsing, -v/-q, bucket status, and determine_flavors unit coverage.
  • Skill command behavior: bundled skill list/install target parsing and layout rules; use the layout helper instead of performing real installs during usability checks.
  • CLI end-to-end candidates: storage ls/du/find/cp/clone, dataset ls/edit/rm, and gc; mark network/object-copying steps as side-effecting unless run in a controlled fixture.
  • Studio candidates: auth login/logout/token/team, dataset Studio edit/remove, job run/log/cancel/clusters, reconnect behavior, checkpoint reuse, and scheduled-job behavior with mocked Studio responses.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351