automation-pipelines
Build and debug ClearML automation workflows with PipelineController, PipelineDecorator, HPO, schedulers, triggers, and clearml-param-search.
- License 许可
- Apache-2.0
- 最近更新
- 2026-10-08
- Stars
- ★ 11
- Forks
- 3
01 前言 Front Matter
- name
- automation-pipelines
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Build and debug ClearML automation workflows with PipelineController, PipelineDecorator, HPO, schedulers, triggers, and clearml-param-search.
- disable-model-invocation
- true
02 资源文件 4
- 参考文档references/api-reference.md18.0 KB
- 参考文档references/hpo-cli-reference.md6.4 KB
- 参考文档references/troubleshooting.md9.5 KB
- 参考文档references/workflows.md13.0 KB
03 SKILL.md 正文
ClearML Automation Pipelines
Use this sub-skill when an agent needs to create or troubleshoot ClearML automation workflows: pipeline DAGs, decorated function pipelines, hyperparameter search, recurring schedules, event triggers, and automation controller/service tasks.
Read First
- API reference lists the verified automation classes, important signatures, search-space classes, scheduler/trigger calls, and decision notes.
- Workflow recipes shows function pipelines, task pipelines, local debugging, HPO, scheduler, and trigger patterns.
- HPO CLI reference explains
clearml-param-searchflags and JSON payloads, plus offline validation with hpo_search_space_check.py. - Troubleshooting covers parent references, parameter overrides, queues/agents, caching, objective metric matching, JSON errors, and recurring time validation.
- Pipeline skeleton generator writes minimal
PipelineDecoratororPipelineControllerstarter files without contacting a ClearML server.
Route Boundaries
- Use this sub-skill for
PipelineController,PipelineDecorator,HyperParameterOptimizer,TaskScheduler,TriggerScheduler, and theclearml-param-searchCLI. - For logging inside step code with
Task.init,Logger.report_scalar, text, images, tables, or artifacts, route to../experiment-tracking/SKILL.md. - For
Dataset,StorageManager, dataset lineage, or object storage assets passed through pipelines, route to../data-storage/SKILL.md. - For direct
clearml-taskpackaging, enqueueing, or command-line task launches outside HPO/pipeline orchestration, route to../remote-execution-cli/SKILL.md. - For HTTP routers,
Task.get_http_router(), web services, or endpoint deployment, route to../routers-services/SKILL.md.
Automation Decision Path
- Choose
PipelineDecoratorwhen the user has ordinary Python functions and wants pipeline steps created from function calls. - Choose
PipelineController.add_function_step()when the user wants explicit DAG nodes from Python functions, named return artifacts, parameter references, callbacks, retries, or queue overrides. - Choose
PipelineController.add_step()when the user already has ClearML Tasks to clone and connect into a DAG. - Choose
HyperParameterOptimizerorclearml-param-searchwhen the user has a base task/script and wants to clone it across parameter ranges using an objective scalar. - Choose
TaskSchedulerfor recurring cron-like launches andTriggerSchedulerfor model, dataset, or task events.
Safety And Execution Notes
- Draft pipeline/HPO/scheduler code without running it unless the user explicitly has a configured ClearML server, credentials, queues, and safe execution target.
- Prefer local debugging paths first:
PipelineDecorator.debug_pipeline(),PipelineDecorator.run_locally(),PipelineController.start_locally(), or HPO--localwhere appropriate. - Treat remote queue names as deployment choices: pipeline controller defaults to a
servicesqueue, while step execution and HPO jobs usually need worker queues. - Keep pipeline step imports inside the step function when they are required remotely; ClearML can infer per-step packages from imports used in function steps.
04 同仓库更多 Skills
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
python-api
Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP/mAP semantics, and source-style import diagnostics.
file-evaluation
Folder-based PASCAL VOC AP/mAP evaluation from ground-truth and detection text files.
object-detection-metrics
Guides legacy Object-Detection-Metrics PASCAL VOC AP/mAP evaluation from detection text folders and direct Python API classes.
nx-workspace-patterns
Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.