[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6ACD7C60DBKTG3CQ9936QDX":3,"skill-same:01M6ACD7C60DBKTG3CQ9936QDX":81},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":41,"linkOnly":42,"markdownContent":43,"markdownBody":44,"frontMatter":45,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":49,"repositoryHtmlUrl":50,"repositoryHomepageUrl":51,"ownerHtmlUrl":52,"ownerBlogUrl":53,"ownerDescription":6,"updatedAt":54,"install":55,"resources":68},"01M6ACD7C60DBKTG3CQ9936QDX","ci-cd",null,"Guides understanding and working with Apache Beam's CI\u002FCD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration.","apache\u002Fbeam","apache","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F47359?v=4","01M6AC83E6K4FDKK7HZ8WNMQKE","The Apache Software Foundation","01M6ACCXED4SMY85J25QX8K9SY",8718,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.agent\u002Fskills\u002Fci-cd\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000011","运维与部署","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,9,37,38,39,40],"ci cd","cd","ci","github actions","when","with","CI\u002FCD","guides","configuration","use","system","beam","actions","debugging","workflows","test","github","2026-11-10 10:12:30",false,"---\n# Licensed to the Apache Software Foundation (ASF) under one\n# or more contributor license agreements.  See the NOTICE file\n# distributed with this work for additional information\n# regarding copyright ownership.  The ASF licenses this file\n# to you under the Apache License, Version 2.0 (the\n# \"License\"); you may not use this file except in compliance\n# with the License.  You may obtain a copy of the License at\n#\n#   http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing,\n# software distributed under the License is distributed on an\n# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n# KIND, either express or implied.  See the License for the\n# specific language governing permissions and limitations\n# under the License.\n\nname: ci-cd\ndescription: Guides understanding and working with Apache Beam's CI\u002FCD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration.\n---\n\n# CI\u002FCD in Apache Beam\n\n## Overview\nApache Beam uses GitHub Actions for CI\u002FCD. Workflows are located in `.github\u002Fworkflows\u002F`.\n\n## Workflow Types\n\n### PreCommit Workflows\n- Run on PRs and merges\n- Validate code changes before merge\n- Naming: `beam_PreCommit_*.yml`\n\n### PostCommit Workflows\n- Run after merge and on schedule\n- More comprehensive testing\n- Naming: `beam_PostCommit_*.yml`\n\n### Scheduled Workflows\n- Run nightly on master\n- Check for external dependency impacts\n- Tag master with `nightly-master`\n\n## Key Workflows\n\n### PreCommit\n| Workflow | Description |\n|----------|-------------|\n| `beam_PreCommit_Java.yml` | Java build and tests |\n| `beam_PreCommit_Python.yml` | Python tests |\n| `beam_PreCommit_Go.yml` | Go tests |\n| `beam_PreCommit_RAT.yml` | License header checks |\n| `beam_PreCommit_Spotless.yml` | Code formatting |\n\n### PostCommit - Java\n| Workflow | Description |\n|----------|-------------|\n| `beam_PostCommit_Java.yml` | Full Java test suite |\n| `beam_PostCommit_Java_ValidatesRunner_*.yml` | Runner validation tests |\n| `beam_PostCommit_Java_Examples_*.yml` | Example pipeline tests |\n\n### PostCommit - Python\n| Workflow | Description |\n|----------|-------------|\n| `beam_PostCommit_Python.yml` | Full Python test suite |\n| `beam_PostCommit_Python_ValidatesRunner_*.yml` | Runner validation |\n| `beam_PostCommit_Python_Examples_*.yml` | Examples |\n\n### Load & Performance Tests\n| Workflow | Description |\n|----------|-------------|\n| `beam_LoadTests_*.yml` | Load testing |\n| `beam_PerformanceTests_*.yml` | I\u002FO performance |\n\n## Triggering Tests\n\n### Automatic\n- PRs trigger PreCommit tests\n- Merges trigger PostCommit tests\n\n### Triggering Specific Workflows\nUse [trigger files](https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.github\u002Fworkflows\u002FREADME.md#running-workflows-manually) to run specific workflows.\n\n### Workflow Dispatch\nMost workflows support manual triggering via GitHub UI.\n\n## Understanding Test Results\n\n### Finding Logs\n1. Go to PR → Checks tab\n2. Click on failed workflow\n3. Expand failed job\n4. View step logs\n\n### Common Failure Patterns\n\n#### Flaky Tests\n- Random failures unrelated to change\n- Solution: Use [trigger files](https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.github\u002Fworkflows\u002FREADME.md#running-workflows-manually) to re-run the specific workflow.\n\n#### Timeout\n- Increase timeout in workflow if justified\n- Or optimize test\n\n#### Resource Exhaustion\n- GCP quota issues\n- Check project settings\n\n## GCP Credentials\n\nWorkflows requiring GCP access use these secrets:\n- `GCP_PROJECT_ID` - Project ID (e.g., `apache-beam-testing`)\n- `GCP_REGION` - Region (e.g., `us-central1`)\n- `GCP_TESTING_BUCKET` - Temp storage bucket\n- `GCP_PYTHON_WHEELS_BUCKET` - Python wheels bucket\n- `GCP_SA_EMAIL` - Service account email\n- `GCP_SA_KEY` - Base64-encoded service account key\n\nRequired IAM roles:\n- Storage Admin\n- Dataflow Admin\n- Artifact Registry Writer\n- BigQuery Data Editor\n- Service Account User\n\n## Self-hosted vs GitHub-hosted Runners\n\n### Self-hosted (majority of workflows)\n- Pre-configured with dependencies\n- GCP credentials pre-configured\n- Naming: `beam_*.yml`\n\n### GitHub-hosted\n- Used for cross-platform testing (Linux, macOS, Windows)\n- May need explicit credential setup\n\n## Workflow Structure\n\n```yaml\nname: Workflow Name\non:\n  push:\n    branches: [master]\n  pull_request:\n    branches: [master]\n  schedule:\n    - cron: '0 0 * * *'\n  workflow_dispatch:\n\njobs:\n  build:\n    runs-on: [self-hosted, ...]\n    steps:\n      - uses: actions\u002Fcheckout@v4\n      - name: Run Gradle\n        run: .\u002Fgradlew :task:name\n```\n\n## Local Debugging\n\n### Run Same Commands as CI\nCheck workflow file's `run` commands:\n```bash\n.\u002Fgradlew :sdks:java:core:test\n.\u002Fgradlew :sdks:python:test\n```\n\n### Common Issues\n- Clean gradle cache: `rm -rf ~\u002F.gradle .gradle`\n- Remove build directory: `rm -rf build`\n- Check Java version matches CI\n\n## Snapshot Builds\n\n### Locations\n- Java SDK: https:\u002F\u002Frepository.apache.org\u002Fcontent\u002Fgroups\u002Fsnapshots\u002Forg\u002Fapache\u002Fbeam\u002F\n- SDK Containers: https:\u002F\u002Fgcr.io\u002Fapache-beam-testing\u002Fbeam-sdk\n- Portable Runners: https:\u002F\u002Fgcr.io\u002Fapache-beam-testing\u002Fbeam_portability\n- Python SDK: gs:\u002F\u002Fbeam-python-nightly-snapshots\n\n## Release Workflows\n| Workflow | Purpose |\n|----------|---------|\n| `cut_release_branch.yml` | Create release branch |\n| `build_release_candidate.yml` | Build RC |\n| `finalize_release.yml` | Finalize release |\n| `publish_github_release_notes.yml` | Publish notes |\n","# CI\u002FCD in Apache Beam\n\n## Overview\nApache Beam uses GitHub Actions for CI\u002FCD. Workflows are located in `.github\u002Fworkflows\u002F`.\n\n## Workflow Types\n\n### PreCommit Workflows\n- Run on PRs and merges\n- Validate code changes before merge\n- Naming: `beam_PreCommit_*.yml`\n\n### PostCommit Workflows\n- Run after merge and on schedule\n- More comprehensive testing\n- Naming: `beam_PostCommit_*.yml`\n\n### Scheduled Workflows\n- Run nightly on master\n- Check for external dependency impacts\n- Tag master with `nightly-master`\n\n## Key Workflows\n\n### PreCommit\n| Workflow | Description |\n|----------|-------------|\n| `beam_PreCommit_Java.yml` | Java build and tests |\n| `beam_PreCommit_Python.yml` | Python tests |\n| `beam_PreCommit_Go.yml` | Go tests |\n| `beam_PreCommit_RAT.yml` | License header checks |\n| `beam_PreCommit_Spotless.yml` | Code formatting |\n\n### PostCommit - Java\n| Workflow | Description |\n|----------|-------------|\n| `beam_PostCommit_Java.yml` | Full Java test suite |\n| `beam_PostCommit_Java_ValidatesRunner_*.yml` | Runner validation tests |\n| `beam_PostCommit_Java_Examples_*.yml` | Example pipeline tests |\n\n### PostCommit - Python\n| Workflow | Description |\n|----------|-------------|\n| `beam_PostCommit_Python.yml` | Full Python test suite |\n| `beam_PostCommit_Python_ValidatesRunner_*.yml` | Runner validation |\n| `beam_PostCommit_Python_Examples_*.yml` | Examples |\n\n### Load & Performance Tests\n| Workflow | Description |\n|----------|-------------|\n| `beam_LoadTests_*.yml` | Load testing |\n| `beam_PerformanceTests_*.yml` | I\u002FO performance |\n\n## Triggering Tests\n\n### Automatic\n- PRs trigger PreCommit tests\n- Merges trigger PostCommit tests\n\n### Triggering Specific Workflows\nUse [trigger files](https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.github\u002Fworkflows\u002FREADME.md#running-workflows-manually) to run specific workflows.\n\n### Workflow Dispatch\nMost workflows support manual triggering via GitHub UI.\n\n## Understanding Test Results\n\n### Finding Logs\n1. Go to PR → Checks tab\n2. Click on failed workflow\n3. Expand failed job\n4. View step logs\n\n### Common Failure Patterns\n\n#### Flaky Tests\n- Random failures unrelated to change\n- Solution: Use [trigger files](https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.github\u002Fworkflows\u002FREADME.md#running-workflows-manually) to re-run the specific workflow.\n\n#### Timeout\n- Increase timeout in workflow if justified\n- Or optimize test\n\n#### Resource Exhaustion\n- GCP quota issues\n- Check project settings\n\n## GCP Credentials\n\nWorkflows requiring GCP access use these secrets:\n- `GCP_PROJECT_ID` - Project ID (e.g., `apache-beam-testing`)\n- `GCP_REGION` - Region (e.g., `us-central1`)\n- `GCP_TESTING_BUCKET` - Temp storage bucket\n- `GCP_PYTHON_WHEELS_BUCKET` - Python wheels bucket\n- `GCP_SA_EMAIL` - Service account email\n- `GCP_SA_KEY` - Base64-encoded service account key\n\nRequired IAM roles:\n- Storage Admin\n- Dataflow Admin\n- Artifact Registry Writer\n- BigQuery Data Editor\n- Service Account User\n\n## Self-hosted vs GitHub-hosted Runners\n\n### Self-hosted (majority of workflows)\n- Pre-configured with dependencies\n- GCP credentials pre-configured\n- Naming: `beam_*.yml`\n\n### GitHub-hosted\n- Used for cross-platform testing (Linux, macOS, Windows)\n- May need explicit credential setup\n\n## Workflow Structure\n\n```yaml\nname: Workflow Name\non:\n  push:\n    branches: [master]\n  pull_request:\n    branches: [master]\n  schedule:\n    - cron: '0 0 * * *'\n  workflow_dispatch:\n\njobs:\n  build:\n    runs-on: [self-hosted, ...]\n    steps:\n      - uses: actions\u002Fcheckout@v4\n      - name: Run Gradle\n        run: .\u002Fgradlew :task:name\n```\n\n## Local Debugging\n\n### Run Same Commands as CI\nCheck workflow file's `run` commands:\n```bash\n.\u002Fgradlew :sdks:java:core:test\n.\u002Fgradlew :sdks:python:test\n```\n\n### Common Issues\n- Clean gradle cache: `rm -rf ~\u002F.gradle .gradle`\n- Remove build directory: `rm -rf build`\n- Check Java version matches CI\n\n## Snapshot Builds\n\n### Locations\n- Java SDK: https:\u002F\u002Frepository.apache.org\u002Fcontent\u002Fgroups\u002Fsnapshots\u002Forg\u002Fapache\u002Fbeam\u002F\n- SDK Containers: https:\u002F\u002Fgcr.io\u002Fapache-beam-testing\u002Fbeam-sdk\n- Portable Runners: https:\u002F\u002Fgcr.io\u002Fapache-beam-testing\u002Fbeam_portability\n- Python SDK: gs:\u002F\u002Fbeam-python-nightly-snapshots\n\n## Release Workflows\n| Workflow | Purpose |\n|----------|---------|\n| `cut_release_branch.yml` | Create release branch |\n| `build_release_candidate.yml` | Build RC |\n| `finalize_release.yml` | Finalize release |\n| `publish_github_release_notes.yml` | Publish notes |\n",{"name":5,"description":7},"master","aba9e790548a6870a46d2c085c817c2754f4a301",".agent\u002Fskills\u002Fci-cd",4810,"https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam","https:\u002F\u002Fbeam.apache.org\u002F","https:\u002F\u002Fgithub.com\u002Fapache","https:\u002F\u002Fwww.apache.org\u002F","2026-11-14 06:08:33",{"source":8,"skillName":5,"commands":56,"prompt":66,"zipAvailable":67},[57,60,63],{"tool":58,"command":59},"npx","npx skills add apache\u002Fbeam --skill ci-cd",{"tool":61,"command":62},"bunx","bunx skills add apache\u002Fbeam --skill ci-cd",{"tool":64,"command":65},"pnpm","pnpm dlx skills add apache\u002Fbeam --skill ci-cd","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.agent\u002Fskills\u002Fci-cd\u002FSKILL.md",true,[69,75],{"path":70,"resourceType":71,"resourceTypeDesc":72,"sizeBytes":73,"sourceHtmlUrl":74},"assets\u002FSYSTEM_TEMPLATE","ASSET","资产文件",1183,"https:\u002F\u002Fgithub.com\u002Fgithub\u002Fawesome-copilot\u002Fblob\u002Fmain\u002Fskills\u002Fnano-banana-pro-openrouter\u002Fassets\u002FSYSTEM_TEMPLATE",{"path":76,"resourceType":77,"resourceTypeDesc":78,"sizeBytes":79,"sourceHtmlUrl":80},"scripts\u002Fgenerate_image.py","SCRIPT","脚本",5926,"https:\u002F\u002Fgithub.com\u002Fgithub\u002Fawesome-copilot\u002Fblob\u002Fmain\u002Fskills\u002Fnano-banana-pro-openrouter\u002Fscripts\u002Fgenerate_image.py",[82,101,117,131,151,165],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":41},"01M6ACDYBTKYKDP9Z0BQKF7XMZ","yaml-development","Guides YAML SDK development in Apache Beam, including environment setup, testing, and key concepts. Use when working with Beam YAML code in sdks\u002Fpython\u002Fapache_beam\u002Fyaml\u002F.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.agent\u002Fskills\u002Fyaml-development\u002FSKILL.md",[],[89,90,91,92,35,93,94,95,96,9,97,33,31,29,98,99,100,28],"development","yaml development","yaml","testing","python","sdk","setup","code","environment","environment setup","concepts","sdks",{"skillNo":102,"name":103,"nameZh":6,"description":104,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":105,"categories":106,"keywords":107,"publishedAt":41},"01M6ACDVK2M4QNJ2T0423H23SY","runners","Guides understanding and working with Apache Beam runners (Direct, Dataflow, Flink, Spark, etc.). Use when configuring pipelines for different execution environments or debugging runner-specific issues.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.agent\u002Fskills\u002Frunners\u002FSKILL.md",[],[103,35,108,37,109,9,110,111,112,33,113,114,31,29,115,116,28],"spark","issues","flink","execution","direct","pipelines","runner","environments","for",{"skillNo":118,"name":119,"nameZh":6,"description":120,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":121,"categories":122,"keywords":126,"publishedAt":41},"01M6ACDRVH48A1WBYH9DEETW7F","python-development","Guides Python SDK development in Apache Beam, including environment setup, testing, building, and running pipelines. Use when working with Python code in sdks\u002Fpython\u002F.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.agent\u002Fskills\u002Fpython-development\u002FSKILL.md",[123],{"categoryNo":124,"categoryName":125,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",[93,127,89,92,35,94,95,96,9,128,97,33,113,31,29,98,129,100,28,130],"python development","python sdk","python code","running",{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":135,"categories":136,"keywords":142,"publishedAt":41},"01M6ACDPYBMSHSQNCFXVJ7VJW3","license-compliance","Ensures all new files include proper Apache 2.0 license headers. Use when creating any new file in the Apache Beam repository.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.agent\u002Fskills\u002Flicense-compliance\u002FSKILL.md",[137],{"categoryNo":138,"categoryName":139,"taxonomyType":140,"taxonomyTypeDesc":141},"01JBSEEDCAT000000000000037","律师与法务","PROFESSION","适用职业",[143,144,145,35,9,146,147,33,148,149,28,150],"license","license compliance","compliance","file","repository","files","headers","new",{"skillNo":152,"name":153,"nameZh":6,"description":154,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":155,"categories":156,"keywords":158,"publishedAt":41},"01M6ACDM2W664NC4FH95B3NC14","java-development","Guides Java SDK development in Apache Beam, including building, testing, running examples, and understanding the project structure. Use when working with Java code in sdks\u002Fjava\u002F, runners\u002F, or examples\u002Fjava\u002F.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.agent\u002Fskills\u002Fjava-development\u002FSKILL.md",[157],{"categoryNo":124,"categoryName":125,"taxonomyType":21,"taxonomyTypeDesc":22},[89,159,160,92,35,94,161,96,9,103,162,33,163,164,31,29,100,28,130],"java","java development","project structure","examples","structure","project",{"skillNo":166,"name":167,"nameZh":6,"description":168,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":169,"categories":170,"keywords":172,"publishedAt":41},"01M6ACDHGVH4F8ADMCFR7T34H8","io-connectors","Guides development and usage of I\u002FO connectors in Apache Beam. Use when working with I\u002FO connectors, creating new connectors, or debugging data source\u002Fsink issues.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fbeam\u002Fblob\u002Fmaster\u002F.agent\u002Fskills\u002Fio-connectors\u002FSKILL.md",[171],{"categoryNo":124,"categoryName":125,"taxonomyType":21,"taxonomyTypeDesc":22},[173,174,175,35,37,89,109,9,176,33,31,29,177,178,28,150],"connectors","io connectors","io","data","usage","source"]