[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6FSN56GZYRXC6JQJGR0XM4Z":3,"skill-same:01M6FSN56GZYRXC6JQJGR0XM4Z":71},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":49,"commitSha":50,"skillDirectory":51,"forksCount":14,"repositoryHtmlUrl":52,"repositoryHomepageUrl":6,"ownerHtmlUrl":53,"ownerBlogUrl":54,"ownerDescription":55,"updatedAt":56,"install":57,"resources":70},"01M6FSN56GZYRXC6JQJGR0XM4Z","ml-pipeline-workflow",null,"Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating mod...","reikiplanet\u002Fdaoskills","reikiplanet","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F96377599?v=4","01M6CVWCMD740YHQDK7E9QYBW4","Reiki Planet","01M6CVWCS4KXVA2CWFVJ43EAXK",0,"MIT","https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills\u002Fblob\u002Fmaster\u002Fautomation\u002Fml-pipeline-workflow\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"ml","workflow","ml pipeline","ml pipeline workflow","pipeline","ML Pipelines","end","community","build","pipelines","use","data preparation","model","Production","data","validation","mlops","deployment","training","2026-11-10 10:12:30",false,"---\nname: ml-pipeline-workflow\ndescription: \"Build end-to-end MLOps pipelines from data preparation through model training, validation, and production deployment. Use when creating ML pipelines, implementing MLOps practices, or automating mod...\"\nrisk: unknown\nsource: community\n---\n\n# ML Pipeline Workflow\n\nComplete end-to-end MLOps pipeline orchestration from data preparation through model deployment.\n\n## Do not use this skill when\n\n- The task is unrelated to ml pipeline workflow\n- You need a different domain or tool outside this scope\n\n## Instructions\n\n- Clarify goals, constraints, and required inputs.\n- Apply relevant best practices and validate outcomes.\n- Provide actionable steps and verification.\n- If detailed examples are required, open `resources\u002Fimplementation-playbook.md`.\n\n## Overview\n\nThis skill provides comprehensive guidance for building production ML pipelines that handle the full lifecycle: data ingestion → preparation → training → validation → deployment → monitoring.\n\n## Use this skill when\n\n- Building new ML pipelines from scratch\n- Designing workflow orchestration for ML systems\n- Implementing data → model → deployment automation\n- Setting up reproducible training workflows\n- Creating DAG-based ML orchestration\n- Integrating ML components into production systems\n\n## What This Skill Provides\n\n### Core Capabilities\n\n1. **Pipeline Architecture**\n   - End-to-end workflow design\n   - DAG orchestration patterns (Airflow, Dagster, Kubeflow)\n   - Component dependencies and data flow\n   - Error handling and retry strategies\n\n2. **Data Preparation**\n   - Data validation and quality checks\n   - Feature engineering pipelines\n   - Data versioning and lineage\n   - Train\u002Fvalidation\u002Ftest splitting strategies\n\n3. **Model Training**\n   - Training job orchestration\n   - Hyperparameter management\n   - Experiment tracking integration\n   - Distributed training patterns\n\n4. **Model Validation**\n   - Validation frameworks and metrics\n   - A\u002FB testing infrastructure\n   - Performance regression detection\n   - Model comparison workflows\n\n5. **Deployment Automation**\n   - Model serving patterns\n   - Canary deployments\n   - Blue-green deployment strategies\n   - Rollback mechanisms\n\n### Reference Documentation\n\nSee the `references\u002F` directory for detailed guides:\n- **data-preparation.md** - Data cleaning, validation, and feature engineering\n- **model-training.md** - Training workflows and best practices\n- **model-validation.md** - Validation strategies and metrics\n- **model-deployment.md** - Deployment patterns and serving architectures\n\n### Assets and Templates\n\nThe `assets\u002F` directory contains:\n- **pipeline-dag.yaml.template** - DAG template for workflow orchestration\n- **training-config.yaml** - Training configuration template\n- **validation-checklist.md** - Pre-deployment validation checklist\n\n## Usage Patterns\n\n### Basic Pipeline Setup\n\n```python\n# 1. Define pipeline stages\nstages = [\n    \"data_ingestion\",\n    \"data_validation\",\n    \"feature_engineering\",\n    \"model_training\",\n    \"model_validation\",\n    \"model_deployment\"\n]\n\n# 2. Configure dependencies\n# See assets\u002Fpipeline-dag.yaml.template for full example\n```\n\n### Production Workflow\n\n1. **Data Preparation Phase**\n   - Ingest raw data from sources\n   - Run data quality checks\n   - Apply feature transformations\n   - Version processed datasets\n\n2. **Training Phase**\n   - Load versioned training data\n   - Execute training jobs\n   - Track experiments and metrics\n   - Save trained models\n\n3. **Validation Phase**\n   - Run validation test suite\n   - Compare against baseline\n   - Generate performance reports\n   - Approve for deployment\n\n4. **Deployment Phase**\n   - Package model artifacts\n   - Deploy to serving infrastructure\n   - Configure monitoring\n   - Validate production traffic\n\n## Best Practices\n\n### Pipeline Design\n\n- **Modularity**: Each stage should be independently testable\n- **Idempotency**: Re-running stages should be safe\n- **Observability**: Log metrics at every stage\n- **Versioning**: Track data, code, and model versions\n- **Failure Handling**: Implement retry logic and alerting\n\n### Data Management\n\n- Use data validation libraries (Great Expectations, TFX)\n- Version datasets with DVC or similar tools\n- Document feature engineering transformations\n- Maintain data lineage tracking\n\n### Model Operations\n\n- Separate training and serving infrastructure\n- Use model registries (MLflow, Weights & Biases)\n- Implement gradual rollouts for new models\n- Monitor model performance drift\n- Maintain rollback capabilities\n\n### Deployment Strategies\n\n- Start with shadow deployments\n- Use canary releases for validation\n- Implement A\u002FB testing infrastructure\n- Set up automated rollback triggers\n- Monitor latency and throughput\n\n## Integration Points\n\n### Orchestration Tools\n\n- **Apache Airflow**: DAG-based workflow orchestration\n- **Dagster**: Asset-based pipeline orchestration\n- **Kubeflow Pipelines**: Kubernetes-native ML workflows\n- **Prefect**: Modern dataflow automation\n\n### Experiment Tracking\n\n- MLflow for experiment tracking and model registry\n- Weights & Biases for visualization and collaboration\n- TensorBoard for training metrics\n\n### Deployment Platforms\n\n- AWS SageMaker for managed ML infrastructure\n- Google Vertex AI for GCP deployments\n- Azure ML for Azure cloud\n- Kubernetes + KServe for cloud-agnostic serving\n\n## Progressive Disclosure\n\nStart with the basics and gradually add complexity:\n\n1. **Level 1**: Simple linear pipeline (data → train → deploy)\n2. **Level 2**: Add validation and monitoring stages\n3. **Level 3**: Implement hyperparameter tuning\n4. **Level 4**: Add A\u002FB testing and gradual rollouts\n5. **Level 5**: Multi-model pipelines with ensemble strategies\n\n## Common Patterns\n\n### Batch Training Pipeline\n\n```yaml\n# See assets\u002Fpipeline-dag.yaml.template\nstages:\n  - name: data_preparation\n    dependencies: []\n  - name: model_training\n    dependencies: [data_preparation]\n  - name: model_evaluation\n    dependencies: [model_training]\n  - name: model_deployment\n    dependencies: [model_evaluation]\n```\n\n### Real-time Feature Pipeline\n\n```python\n# Stream processing for real-time features\n# Combined with batch training\n# See references\u002Fdata-preparation.md\n```\n\n### Continuous Training\n\n```python\n# Automated retraining on schedule\n# Triggered by data drift detection\n# See references\u002Fmodel-training.md\n```\n\n## Troubleshooting\n\n### Common Issues\n\n- **Pipeline failures**: Check dependencies and data availability\n- **Training instability**: Review hyperparameters and data quality\n- **Deployment issues**: Validate model artifacts and serving config\n- **Performance degradation**: Monitor data drift and model metrics\n\n### Debugging Steps\n\n1. Check pipeline logs for each stage\n2. Validate input\u002Foutput data at boundaries\n3. Test components in isolation\n4. Review experiment tracking metrics\n5. Inspect model artifacts and metadata\n\n## Next Steps\n\nAfter setting up your pipeline:\n\n1. Explore **hyperparameter-tuning** skill for optimization\n2. Learn **experiment-tracking-setup** for MLflow\u002FW&B\n3. Review **model-deployment-patterns** for serving strategies\n4. Implement monitoring with observability tools\n\n## Related Skills\n\n- **experiment-tracking-setup**: MLflow and Weights & Biases integration\n- **hyperparameter-tuning**: Automated hyperparameter optimization\n- **model-deployment-patterns**: Advanced deployment strategies\n","# ML Pipeline Workflow\n\nComplete end-to-end MLOps pipeline orchestration from data preparation through model deployment.\n\n## Do not use this skill when\n\n- The task is unrelated to ml pipeline workflow\n- You need a different domain or tool outside this scope\n\n## Instructions\n\n- Clarify goals, constraints, and required inputs.\n- Apply relevant best practices and validate outcomes.\n- Provide actionable steps and verification.\n- If detailed examples are required, open `resources\u002Fimplementation-playbook.md`.\n\n## Overview\n\nThis skill provides comprehensive guidance for building production ML pipelines that handle the full lifecycle: data ingestion → preparation → training → validation → deployment → monitoring.\n\n## Use this skill when\n\n- Building new ML pipelines from scratch\n- Designing workflow orchestration for ML systems\n- Implementing data → model → deployment automation\n- Setting up reproducible training workflows\n- Creating DAG-based ML orchestration\n- Integrating ML components into production systems\n\n## What This Skill Provides\n\n### Core Capabilities\n\n1. **Pipeline Architecture**\n   - End-to-end workflow design\n   - DAG orchestration patterns (Airflow, Dagster, Kubeflow)\n   - Component dependencies and data flow\n   - Error handling and retry strategies\n\n2. **Data Preparation**\n   - Data validation and quality checks\n   - Feature engineering pipelines\n   - Data versioning and lineage\n   - Train\u002Fvalidation\u002Ftest splitting strategies\n\n3. **Model Training**\n   - Training job orchestration\n   - Hyperparameter management\n   - Experiment tracking integration\n   - Distributed training patterns\n\n4. **Model Validation**\n   - Validation frameworks and metrics\n   - A\u002FB testing infrastructure\n   - Performance regression detection\n   - Model comparison workflows\n\n5. **Deployment Automation**\n   - Model serving patterns\n   - Canary deployments\n   - Blue-green deployment strategies\n   - Rollback mechanisms\n\n### Reference Documentation\n\nSee the `references\u002F` directory for detailed guides:\n- **data-preparation.md** - Data cleaning, validation, and feature engineering\n- **model-training.md** - Training workflows and best practices\n- **model-validation.md** - Validation strategies and metrics\n- **model-deployment.md** - Deployment patterns and serving architectures\n\n### Assets and Templates\n\nThe `assets\u002F` directory contains:\n- **pipeline-dag.yaml.template** - DAG template for workflow orchestration\n- **training-config.yaml** - Training configuration template\n- **validation-checklist.md** - Pre-deployment validation checklist\n\n## Usage Patterns\n\n### Basic Pipeline Setup\n\n```python\n# 1. Define pipeline stages\nstages = [\n    \"data_ingestion\",\n    \"data_validation\",\n    \"feature_engineering\",\n    \"model_training\",\n    \"model_validation\",\n    \"model_deployment\"\n]\n\n# 2. Configure dependencies\n# See assets\u002Fpipeline-dag.yaml.template for full example\n```\n\n### Production Workflow\n\n1. **Data Preparation Phase**\n   - Ingest raw data from sources\n   - Run data quality checks\n   - Apply feature transformations\n   - Version processed datasets\n\n2. **Training Phase**\n   - Load versioned training data\n   - Execute training jobs\n   - Track experiments and metrics\n   - Save trained models\n\n3. **Validation Phase**\n   - Run validation test suite\n   - Compare against baseline\n   - Generate performance reports\n   - Approve for deployment\n\n4. **Deployment Phase**\n   - Package model artifacts\n   - Deploy to serving infrastructure\n   - Configure monitoring\n   - Validate production traffic\n\n## Best Practices\n\n### Pipeline Design\n\n- **Modularity**: Each stage should be independently testable\n- **Idempotency**: Re-running stages should be safe\n- **Observability**: Log metrics at every stage\n- **Versioning**: Track data, code, and model versions\n- **Failure Handling**: Implement retry logic and alerting\n\n### Data Management\n\n- Use data validation libraries (Great Expectations, TFX)\n- Version datasets with DVC or similar tools\n- Document feature engineering transformations\n- Maintain data lineage tracking\n\n### Model Operations\n\n- Separate training and serving infrastructure\n- Use model registries (MLflow, Weights & Biases)\n- Implement gradual rollouts for new models\n- Monitor model performance drift\n- Maintain rollback capabilities\n\n### Deployment Strategies\n\n- Start with shadow deployments\n- Use canary releases for validation\n- Implement A\u002FB testing infrastructure\n- Set up automated rollback triggers\n- Monitor latency and throughput\n\n## Integration Points\n\n### Orchestration Tools\n\n- **Apache Airflow**: DAG-based workflow orchestration\n- **Dagster**: Asset-based pipeline orchestration\n- **Kubeflow Pipelines**: Kubernetes-native ML workflows\n- **Prefect**: Modern dataflow automation\n\n### Experiment Tracking\n\n- MLflow for experiment tracking and model registry\n- Weights & Biases for visualization and collaboration\n- TensorBoard for training metrics\n\n### Deployment Platforms\n\n- AWS SageMaker for managed ML infrastructure\n- Google Vertex AI for GCP deployments\n- Azure ML for Azure cloud\n- Kubernetes + KServe for cloud-agnostic serving\n\n## Progressive Disclosure\n\nStart with the basics and gradually add complexity:\n\n1. **Level 1**: Simple linear pipeline (data → train → deploy)\n2. **Level 2**: Add validation and monitoring stages\n3. **Level 3**: Implement hyperparameter tuning\n4. **Level 4**: Add A\u002FB testing and gradual rollouts\n5. **Level 5**: Multi-model pipelines with ensemble strategies\n\n## Common Patterns\n\n### Batch Training Pipeline\n\n```yaml\n# See assets\u002Fpipeline-dag.yaml.template\nstages:\n  - name: data_preparation\n    dependencies: []\n  - name: model_training\n    dependencies: [data_preparation]\n  - name: model_evaluation\n    dependencies: [model_training]\n  - name: model_deployment\n    dependencies: [model_evaluation]\n```\n\n### Real-time Feature Pipeline\n\n```python\n# Stream processing for real-time features\n# Combined with batch training\n# See references\u002Fdata-preparation.md\n```\n\n### Continuous Training\n\n```python\n# Automated retraining on schedule\n# Triggered by data drift detection\n# See references\u002Fmodel-training.md\n```\n\n## Troubleshooting\n\n### Common Issues\n\n- **Pipeline failures**: Check dependencies and data availability\n- **Training instability**: Review hyperparameters and data quality\n- **Deployment issues**: Validate model artifacts and serving config\n- **Performance degradation**: Monitor data drift and model metrics\n\n### Debugging Steps\n\n1. Check pipeline logs for each stage\n2. Validate input\u002Foutput data at boundaries\n3. Test components in isolation\n4. Review experiment tracking metrics\n5. Inspect model artifacts and metadata\n\n## Next Steps\n\nAfter setting up your pipeline:\n\n1. Explore **hyperparameter-tuning** skill for optimization\n2. Learn **experiment-tracking-setup** for MLflow\u002FW&B\n3. Review **model-deployment-patterns** for serving strategies\n4. Implement monitoring with observability tools\n\n## Related Skills\n\n- **experiment-tracking-setup**: MLflow and Weights & Biases integration\n- **hyperparameter-tuning**: Automated hyperparameter optimization\n- **model-deployment-patterns**: Advanced deployment strategies\n",{"name":5,"risk":48,"source":31,"description":7},"unknown","master","dff58085da947e81e8418c922055644bf0082b1b","automation\u002Fml-pipeline-workflow","https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills","https:\u002F\u002Fgithub.com\u002Freikiplanet","https:\u002F\u002Fwww.reiki-planet.com","Reiki Planet Game","2026-03-29 06:02:29",{"source":8,"skillName":5,"commands":58,"prompt":68,"zipAvailable":69},[59,62,65],{"tool":60,"command":61},"npx","npx skills add reikiplanet\u002Fdaoskills --skill ml-pipeline-workflow",{"tool":63,"command":64},"bunx","bunx skills add reikiplanet\u002Fdaoskills --skill ml-pipeline-workflow",{"tool":66,"command":67},"pnpm","pnpm dlx skills add reikiplanet\u002Fdaoskills --skill ml-pipeline-workflow","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills\u002Fblob\u002Fmaster\u002Fautomation\u002Fml-pipeline-workflow\u002FSKILL.md",true,[],[72,98,110,124,144,167],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":43},"01M6G614JTEQJHPT8AKK3YQ18S","web-games","Web browser game development principles. Framework selection, WebGPU, optimization, PWA.","https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills\u002Fblob\u002Fmaster\u002Fweb-games\u002FSKILL.md",[],[79,80,81,82,83,84,31,85,86,87,88,89,90,91,92,93,94,95,96,97],"games","web games","web","grep","principles","selection","framework selection","edit","write","read","pwa","game","web browser","game development","Optimization","framework","browser","development","webgpu",{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":43},"01M6G611H2YGV0GZFP90D38ZMJ","vr-ar","VR\u002FAR development principles. Comfort, interaction, performance requirements.","https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills\u002Fblob\u002Fmaster\u002Fvr-ar\u002FSKILL.md",[],[105,106,96,107,108,31,86,87,88,82,109,83],"ar","vr ar","interaction","requirements","performance",{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":114,"categories":115,"keywords":116,"publishedAt":43},"01M6G60XNMHAMV1SV0EJ1CX0AY","templates","Project scaffolding templates for new applications. Use when creating new projects from scratch. Contains 12 templates for various tech stacks.","https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills\u002Fblob\u002Fmaster\u002Ftemplates\u002FSKILL.md",[],[112,117,118,119,120,121,31,34,88,122,123,82],"project scaffolding","From Scratch","tech","project","scaffolding","projects","applications",{"skillNo":125,"name":126,"nameZh":6,"description":127,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":128,"categories":129,"keywords":130,"publishedAt":43},"01M6G60T0GATMK3Q5H8GDC1Z8S","wcag-audit-patterns","Conduct WCAG 2.2 accessibility audits with automated testing, manual verification, and remediation guidance. Use when auditing websites for accessibility, fixing WCAG violations, or implementing ac...","https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills\u002Fblob\u002Fmaster\u002Fskills\u002Fsecurity\u002Fwcag-audit-patterns\u002FSKILL.md",[],[131,132,133,134,135,31,136,137,138,139,34,140,141,142,143],"wcag audit patterns","wcag audit","patterns","WCAG","audit","manual","with","automated testing","guidance","remediation","verification","accessibility","testing",{"skillNo":145,"name":146,"nameZh":6,"description":147,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":148,"categories":149,"keywords":150,"publishedAt":43},"01M6G60PMCHW1FRZ63F931QK3R","vulnerability-scanner","Advanced vulnerability analysis principles. OWASP 2025, Supply Chain Security, attack surface mapping, risk prioritization.","https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills\u002Fblob\u002Fmaster\u002Fskills\u002Fsecurity\u002Fvulnerability-scanner\u002FSKILL.md",[],[151,152,153,154,155,156,157,158,31,88,82,159,83,160,161,162,163,164,165,166],"scanner","vulnerability scanner","vulnerability","prioritization","2025","mapping","chain","supply chain","Bash","analysis","risk","owasp","supply chain security","security","supply","advanced",{"skillNo":168,"name":169,"nameZh":6,"description":170,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":171,"categories":172,"keywords":173,"publishedAt":43},"01M6G60JZBB8DK0W3DSG1WNMZK","threat-modeling-expert","Expert in threat modeling methodologies, security architecture review, and risk assessment. Masters STRIDE, PASTA, attack trees, and security requirement extraction. Use for security architecture r...","https:\u002F\u002Fgithub.com\u002Freikiplanet\u002Fdaoskills\u002Fblob\u002Fmaster\u002Fskills\u002Fsecurity\u002Fthreat-modeling-expert\u002FSKILL.md",[],[174,175,176,177,178,179,180,181,164,34,182,183,184,31,185,186,187,161,188,189],"modeling","threat modeling expert","threat modeling","expert","threat","risk assessment","architecture review","architecture","security architecture review","trees","security architecture","security requirement extraction","assessment","extraction","review","stride"]