[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7D86MXZCY376Y37HWW309EN":3,"skill-same:01M7D86MXZCY376Y37HWW309EN":82},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":60,"commitSha":61,"skillDirectory":62,"forksCount":14,"repositoryHtmlUrl":63,"repositoryHomepageUrl":6,"ownerHtmlUrl":64,"ownerBlogUrl":65,"ownerDescription":66,"updatedAt":67,"install":68,"resources":81},"01M7D86MXZCY376Y37HWW309EN","CI Pipeline Optimizer",null,"Optimize CI test pipelines through intelligent test splitting, parallelization, caching strategies, and selective test execution based on code changes.","GrupoANDevelopment-m\u002Falberto-ai","GrupoANDevelopment-m","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F257455179?v=4","01M7D69X2N2KEVN0SJF6WGM6SH","Antônio  Ndombe","01M7D69X3P5JTT8H8S904N3888",0,"MIT","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai\u002Fblob\u002Fmain\u002Fskills\u002Fqa\u002Fci-pipeline-optimizer\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: CI Pipeline Optimizer\ndescription: Optimize CI test pipelines through intelligent test splitting, parallelization, caching strategies, and selective test execution based on code changes.\nversion: 1.0.0\nauthor: Pramod\nlicense: MIT\ntags: [ci-cd, pipeline, optimization, parallelization, test-splitting, caching, github-actions]\ntestingTypes: [code-quality]\nframeworks: [jest, vitest, playwright]\nlanguages: [typescript, javascript, python, java]\ndomains: [devops]\nagents: [claude-code, cursor, github-copilot, windsurf, codex, aider, continue, cline, zed, bolt, gemini-cli, amp]\n---\n\n# CI Pipeline Optimizer\n\nSlow CI pipelines are one of the most common productivity killers in software teams. A test suite that takes 30 minutes to run means developers context-switch away from their work, batch multiple changes into fewer PRs to avoid waiting, and eventually start skipping CI altogether. This skill addresses CI pipeline performance through four complementary strategies: intelligent test splitting across parallel workers, selective test execution based on code changes, aggressive caching of dependencies and build artifacts, and pipeline architecture that minimizes total wall-clock time. The techniques apply to any CI system but use GitHub Actions as the primary example, with patterns that transfer to GitLab CI, CircleCI, Jenkins, and other platforms.\n\n## Core Principles\n\n### 1. Wall-Clock Time Is the Only Metric That Matters\n\nCPU time, billable minutes, and total test count are secondary metrics. The developer waits for wall-clock time. A pipeline that uses 60 CPU-minutes across 10 parallel workers in 6 minutes is vastly preferable to a pipeline that uses 20 CPU-minutes sequentially in 20 minutes. Optimize for the duration the developer experiences.\n\n### 2. Never Run Tests You Do Not Need\n\nIf a PR changes only documentation files, running the full test suite is waste. If a PR modifies only the frontend, running backend integration tests is waste. Selective test execution identifies the minimum set of tests needed to validate a specific change, with a safety net that defaults to running everything when the analysis is uncertain.\n\n### 3. Cache Everything That Does Not Change\n\nDependencies, build artifacts, Docker layers, and browser binaries are the same across most CI runs. Downloading and building them from scratch on every run is unnecessary. Aggressive caching can eliminate minutes from every pipeline execution.\n\n### 4. Split Tests by Duration, Not by File Count\n\nNaive test splitting distributes files evenly across workers. But if one file contains a 5-minute test and another contains a 5-second test, the distribution is severely unbalanced. Intelligent splitting uses historical duration data to distribute tests so that all workers finish at approximately the same time.\n\n### 5. Fail Fast, Report Completely\n\nRun the fastest tests first. If unit tests catch a bug in 30 seconds, there is no reason to wait 10 minutes for e2e tests to also catch it. Structure the pipeline so the cheapest checks (lint, type-check) run first to provide the fastest possible feedback, but still collect complete results from all shards for thorough reporting.\n\n## Project Structure\n\n```\nci-config\u002F\n  scripts\u002F\n    split-tests.ts\n    detect-changes.ts\n    cache-manager.ts\n    timing-collector.ts\n    pipeline-analyzer.ts\n  config\u002F\n    test-groups.json\n    change-map.json\n    cache-config.json\n  .github\u002F\n    workflows\u002F\n      ci-optimized.yml\n      ci-selective.yml\n      cache-warmup.yml\n  package.json\n  tsconfig.json\n```\n\nThe `scripts\u002F` directory contains automation tools for test splitting, change detection, cache management, and pipeline analysis. The `config\u002F` directory holds the mapping from file changes to test groups, cache key definitions, and test group specifications.\n\n## Test Splitting Strategies\n\n### Duration-Based Test Splitting\n\nSplit tests across parallel workers based on historical execution times so that each worker runs for approximately the same duration:\n\n```typescript\n\u002F\u002F scripts\u002Fsplit-tests.ts\n\nimport { readFileSync, writeFileSync, existsSync } from 'fs';\nimport { execSync } from 'child_process';\n\ninterface TestTiming {\n  file: string;\n  duration: number;\n  lastRun: string;\n}\n\ninterface SplitResult {\n  shardIndex: number;\n  files: string[];\n  estimatedDuration: number;\n}\n\nexport function splitTestsByDuration(\n  testFiles: string[],\n  shardCount: number,\n  timingsFile: string = 'test-timings.json'\n): SplitResult[] {\n  \u002F\u002F Load historical timings\n  const timings = loadTimings(timingsFile);\n\n  \u002F\u002F Create a list of files with their estimated duration\n  const fileTimings: Array\u003C{ file: string; duration: number }> = testFiles.map((file) => ({\n    file,\n    duration: timings.get(file) || estimateDefaultDuration(file),\n  }));\n\n  \u002F\u002F Sort by duration descending (greedy algorithm: assign largest jobs first)\n  fileTimings.sort((a, b) => b.duration - a.duration);\n\n  \u002F\u002F Initialize shards\n  const shards: SplitResult[] = Array.from({ length: shardCount }, (_, i) => ({\n    shardIndex: i,\n    files: [],\n    estimatedDuration: 0,\n  }));\n\n  \u002F\u002F Greedy assignment: always assign to the shard with the least total duration\n  for (const fileTiming of fileTimings) {\n    const lightest = shards.reduce((min, shard) =>\n      shard.estimatedDuration \u003C min.estimatedDuration ? shard : min\n    );\n    lightest.files.push(fileTiming.file);\n    lightest.estimatedDuration += fileTiming.duration;\n  }\n\n  return shards;\n}\n\nfunction loadTimings(timingsFile: string): Map\u003Cstring, number> {\n  const timings = new Map\u003Cstring, number>();\n\n  if (!existsSync(timingsFile)) {\n    return timings;\n  }\n\n  try {\n    const data: TestTiming[] = JSON.parse(readFileSync(timingsFile, 'utf-8'));\n    for (const entry of data) {\n      timings.set(entry.file, entry.duration);\n    }\n  } catch {\n    console.warn(`Failed to parse timings file: ${timingsFile}`);\n  }\n\n  return timings;\n}\n\nfunction estimateDefaultDuration(file: string): number {\n  \u002F\u002F Heuristic estimates based on test type when no historical data exists\n  if (file.includes('.e2e.') || file.includes('e2e\u002F')) return 30000;\n  if (file.includes('.integration.') || file.includes('integration\u002F')) return 10000;\n  if (file.includes('.spec.')) return 5000;\n  return 3000; \u002F\u002F Default estimate for unit tests\n}\n\n\u002F\u002F CLI entry point for use in GitHub Actions\nfunction main(): void {\n  const shardIndex = parseInt(process.env.SHARD_INDEX || '0', 10);\n  const shardCount = parseInt(process.env.SHARD_COUNT || '1', 10);\n\n  \u002F\u002F Discover all test files\n  const testFilesOutput = execSync(\n    'find . -name \"*.test.ts\" -o -name \"*.spec.ts\" | grep -v node_modules',\n    { encoding: 'utf-8' }\n  );\n  const testFiles = testFilesOutput.trim().split('\\n').filter(Boolean);\n\n  const shards = splitTestsByDuration(testFiles, shardCount);\n  const myShard = shards[shardIndex];\n\n  if (!myShard) {\n    console.error(`Invalid shard index ${shardIndex} for ${shardCount} shards`);\n    process.exit(1);\n  }\n\n  console.log(\n    `Shard ${shardIndex + 1}\u002F${shardCount}: ${myShard.files.length} files, ~${(myShard.estimatedDuration \u002F 1000).toFixed(1)}s`\n  );\n\n  \u002F\u002F Write shard files list for the test runner to consume\n  writeFileSync('shard-files.txt', myShard.files.join('\\n'), 'utf-8');\n}\n\nmain();\n```\n\n### Collecting Test Timings\n\nAfter each CI run, collect and store test execution timings for future split optimization:\n\n```typescript\n\u002F\u002F scripts\u002Ftiming-collector.ts\n\nimport { readFileSync, writeFileSync, existsSync } from 'fs';\n\ninterface JestResult {\n  testResults: Array\u003C{\n    testFilePath: string;\n    perfStats: {\n      runtime: number;\n    };\n  }>;\n}\n\ninterface PlaywrightResult {\n  suites: Array\u003C{\n    file: string;\n    specs: Array\u003C{\n      tests: Array\u003C{\n        results: Array\u003C{\n          duration: number;\n        }>;\n      }>;\n    }>;\n  }>;\n}\n\ninterface TestTiming {\n  file: string;\n  duration: number;\n  lastRun: string;\n}\n\nexport function collectJestTimings(resultsFile: string): TestTiming[] {\n  const results: JestResult = JSON.parse(readFileSync(resultsFile, 'utf-8'));\n\n  return results.testResults.map((test) => ({\n    file: test.testFilePath.replace(process.cwd() + '\u002F', ''),\n    duration: test.perfStats.runtime,\n    lastRun: new Date().toISOString(),\n  }));\n}\n\nexport function collectPlaywrightTimings(resultsFile: string): TestTiming[] {\n  const results: PlaywrightResult = JSON.parse(readFileSync(resultsFile, 'utf-8'));\n  const timings: TestTiming[] = [];\n\n  for (const suite of results.suites) {\n    let totalDuration = 0;\n    for (const spec of suite.specs) {\n      for (const test of spec.tests) {\n        for (const result of test.results) {\n          totalDuration += result.duration;\n        }\n      }\n    }\n\n    timings.push({\n      file: suite.file,\n      duration: totalDuration,\n      lastRun: new Date().toISOString(),\n    });\n  }\n\n  return timings;\n}\n\nexport function mergeTimings(\n  existing: TestTiming[],\n  latest: TestTiming[]\n): TestTiming[] {\n  const map = new Map\u003Cstring, TestTiming>();\n\n  for (const timing of existing) {\n    map.set(timing.file, timing);\n  }\n\n  \u002F\u002F Merge with exponential moving average to smooth out outliers\n  for (const timing of latest) {\n    const prev = map.get(timing.file);\n    if (prev) {\n      \u002F\u002F EMA with alpha = 0.3 gives 70% weight to history, 30% to new data\n      const smoothedDuration = prev.duration * 0.7 + timing.duration * 0.3;\n      map.set(timing.file, {\n        file: timing.file,\n        duration: Math.round(smoothedDuration),\n        lastRun: timing.lastRun,\n      });\n    } else {\n      map.set(timing.file, timing);\n    }\n  }\n\n  return Array.from(map.values());\n}\n\nfunction main(): void {\n  const timingsFile = 'test-timings.json';\n  const existing: TestTiming[] = existsSync(timingsFile)\n    ? JSON.parse(readFileSync(timingsFile, 'utf-8'))\n    : [];\n\n  let latest: TestTiming[] = [];\n\n  if (existsSync('jest-results.json')) {\n    latest = collectJestTimings('jest-results.json');\n  } else if (existsSync('playwright-results.json')) {\n    latest = collectPlaywrightTimings('playwright-results.json');\n  }\n\n  if (latest.length > 0) {\n    const merged = mergeTimings(existing, latest);\n    writeFileSync(timingsFile, JSON.stringify(merged, null, 2), 'utf-8');\n    console.log(`Updated timings for ${latest.length} test files (${merged.length} total)`);\n  } else {\n    console.warn('No test results found to collect timings from');\n  }\n}\n\nmain();\n```\n\n## Selective Test Execution\n\n### Change Detection Engine\n\nDetect which files changed and determine which test groups need to run:\n\n```typescript\n\u002F\u002F scripts\u002Fdetect-changes.ts\n\nimport { execSync } from 'child_process';\nimport { readFileSync, existsSync } from 'fs';\n\ninterface ChangeMap {\n  patterns: Array\u003C{\n    glob: string;\n    testGroups: string[];\n    description: string;\n  }>;\n  testGroups: Record\u003C\n    string,\n    {\n      command: string;\n      files?: string[];\n      description: string;\n    }\n  >;\n}\n\ninterface DetectedChanges {\n  changedFiles: string[];\n  testGroupsToRun: Set\u003Cstring>;\n  skipReason?: string;\n}\n\nexport function detectChanges(baseBranch: string = 'main'): DetectedChanges {\n  let changedFiles: string[];\n\n  try {\n    const diffOutput = execSync(\n      `git diff --name-only origin\u002F${baseBranch}...HEAD`,\n      { encoding: 'utf-8' }\n    );\n    changedFiles = diffOutput.trim().split('\\n').filter(Boolean);\n  } catch {\n    const diffOutput = execSync('git diff --name-only HEAD~1', {\n      encoding: 'utf-8',\n    });\n    changedFiles = diffOutput.trim().split('\\n').filter(Boolean);\n  }\n\n  if (changedFiles.length === 0) {\n    return {\n      changedFiles: [],\n      testGroupsToRun: new Set(),\n      skipReason: 'No files changed',\n    };\n  }\n\n  const changeMap = loadChangeMap();\n  const testGroups = new Set\u003Cstring>();\n\n  for (const file of changedFiles) {\n    for (const pattern of changeMap.patterns) {\n      if (matchGlob(file, pattern.glob)) {\n        for (const group of pattern.testGroups) {\n          testGroups.add(group);\n        }\n      }\n    }\n  }\n\n  \u002F\u002F Safety net: if no patterns matched, run all tests\n  if (testGroups.size === 0) {\n    testGroups.add('all');\n  }\n\n  return {\n    changedFiles,\n    testGroupsToRun: testGroups,\n  };\n}\n\nfunction loadChangeMap(): ChangeMap {\n  const configPath = 'ci-config\u002Fconfig\u002Fchange-map.json';\n  if (!existsSync(configPath)) {\n    return getDefaultChangeMap();\n  }\n  return JSON.parse(readFileSync(configPath, 'utf-8'));\n}\n\nfunction getDefaultChangeMap(): ChangeMap {\n  return {\n    patterns: [\n      {\n        glob: 'src\u002Fapi\u002F**',\n        testGroups: ['unit', 'api-integration'],\n        description: 'API source changes trigger unit and integration tests',\n      },\n      {\n        glob: 'src\u002Fcomponents\u002F**',\n        testGroups: ['unit', 'component'],\n        description: 'UI component changes trigger unit and component tests',\n      },\n      {\n        glob: 'src\u002Fpages\u002F**',\n        testGroups: ['unit', 'e2e'],\n        description: 'Page-level changes trigger unit and E2E tests',\n      },\n      {\n        glob: 'src\u002Flib\u002F**',\n        testGroups: ['unit'],\n        description: 'Library changes trigger unit tests',\n      },\n      {\n        glob: 'src\u002Fdb\u002F**',\n        testGroups: ['unit', 'api-integration', 'e2e'],\n        description: 'Database changes trigger all test types',\n      },\n      {\n        glob: 'package.json',\n        testGroups: ['all'],\n        description: 'Dependency changes require full test run',\n      },\n      {\n        glob: '*.config.*',\n        testGroups: ['all'],\n        description: 'Config changes require full test run',\n      },\n      {\n        glob: '**\u002F*.md',\n        testGroups: ['docs-only'],\n        description: 'Documentation-only changes skip tests',\n      },\n      {\n        glob: '.github\u002F**',\n        testGroups: ['ci-only'],\n        description: 'CI configuration changes need validation',\n      },\n    ],\n    testGroups: {\n      all: { command: 'npm test', description: 'Full test suite' },\n      unit: { command: 'npm run test:unit', description: 'Unit tests only' },\n      'api-integration': {\n        command: 'npm run test:integration',\n        description: 'API integration tests',\n      },\n      component: { command: 'npm run test:components', description: 'Component tests' },\n      e2e: { command: 'npx playwright test', description: 'End-to-end tests' },\n      'docs-only': {\n        command: 'echo \"No tests needed for docs-only changes\"',\n        description: 'Skip tests',\n      },\n      'ci-only': {\n        command: 'echo \"CI config changed - validate workflow syntax only\"',\n        description: 'Validate CI config',\n      },\n    },\n  };\n}\n\nfunction matchGlob(filePath: string, pattern: string): boolean {\n  const regexPattern = pattern\n    .replace(\u002F\\*\\*\u002Fg, '\u003C\u003C\u003CGLOBSTAR>>>')\n    .replace(\u002F\\*\u002Fg, '[^\u002F]*')\n    .replace(\u002F\u003C\u003C\u003CGLOBSTAR>>>\u002Fg, '.*')\n    .replace(\u002F\\?\u002Fg, '.');\n  return new RegExp(`^${regexPattern}$`).test(filePath);\n}\n\nfunction main(): void {\n  const baseBranch = process.env.BASE_BRANCH || 'main';\n  const result = detectChanges(baseBranch);\n\n  console.log(`Changed files: ${result.changedFiles.length}`);\n  console.log(`Test groups to run: ${Array.from(result.testGroupsToRun).join(', ')}`);\n\n  if (result.skipReason) {\n    console.log(`Skip reason: ${result.skipReason}`);\n  }\n\n  \u002F\u002F Output for GitHub Actions\n  const groups = Array.from(result.testGroupsToRun);\n  const shouldRun = groups.length > 0 && !groups.includes('docs-only');\n  console.log(`::set-output name=test-groups::${JSON.stringify(groups)}`);\n  console.log(`::set-output name=should-run-tests::${shouldRun}`);\n}\n\nmain();\n```\n\n## Caching Strategies\n\n### Multi-Layer Cache Configuration\n\n```typescript\n\u002F\u002F scripts\u002Fcache-manager.ts\n\nimport { readFileSync, existsSync } from 'fs';\nimport { createHash } from 'crypto';\n\ninterface CacheLayer {\n  name: string;\n  paths: string[];\n  keyFiles: string[];\n  fallbackKeys: string[];\n  maxAge: number;\n}\n\ninterface CacheConfig {\n  layers: CacheLayer[];\n}\n\nexport function generateCacheKeys(config: CacheConfig): Array\u003C{\n  name: string;\n  key: string;\n  restoreKeys: string[];\n  paths: string[];\n}> {\n  const platform = process.env.RUNNER_OS || process.platform;\n\n  return config.layers.map((layer) => {\n    const fileHashes = layer.keyFiles\n      .filter((f) => existsSync(f))\n      .map((f) => hashFile(f))\n      .join('-');\n\n    const key = `${platform}-${layer.name}-${fileHashes}`;\n\n    const restoreKeys = layer.fallbackKeys.map(\n      (fallback) => `${platform}-${layer.name}-${fallback}`\n    );\n\n    return {\n      name: layer.name,\n      key,\n      restoreKeys,\n      paths: layer.paths,\n    };\n  });\n}\n\nfunction hashFile(filePath: string): string {\n  const content = readFileSync(filePath);\n  return createHash('sha256').update(content).digest('hex').substring(0, 16);\n}\n\nexport const DEFAULT_CACHE_CONFIG: CacheConfig = {\n  layers: [\n    {\n      name: 'node-modules',\n      paths: ['node_modules', '~\u002F.pnpm-store'],\n      keyFiles: ['pnpm-lock.yaml', 'package.json'],\n      fallbackKeys: [''],\n      maxAge: 604800000, \u002F\u002F 7 days\n    },\n    {\n      name: 'playwright-browsers',\n      paths: ['~\u002F.cache\u002Fms-playwright'],\n      keyFiles: ['package.json'],\n      fallbackKeys: [''],\n      maxAge: 2592000000, \u002F\u002F 30 days\n    },\n    {\n      name: 'build-cache',\n      paths: ['.next\u002Fcache', 'dist', '.turbo'],\n      keyFiles: ['tsconfig.json', 'next.config.js'],\n      fallbackKeys: [''],\n      maxAge: 86400000, \u002F\u002F 1 day\n    },\n    {\n      name: 'test-timings',\n      paths: ['test-timings.json'],\n      keyFiles: [],\n      fallbackKeys: [''],\n      maxAge: 2592000000, \u002F\u002F 30 days\n    },\n  ],\n};\n```\n\n## GitHub Actions Optimized Pipeline\n\n### Full Optimized CI Workflow\n\n```yaml\n# .github\u002Fworkflows\u002Fci-optimized.yml\nname: Optimized CI\n\non:\n  push:\n    branches: [main]\n  pull_request:\n    branches: [main]\n\nconcurrency:\n  group: ci-${{ github.ref }}\n  cancel-in-progress: true\n\njobs:\n  # Step 1: Detect changes and determine what to test\n  detect-changes:\n    runs-on: ubuntu-latest\n    outputs:\n      backend: ${{ steps.changes.outputs.backend }}\n      frontend: ${{ steps.changes.outputs.frontend }}\n      config: ${{ steps.changes.outputs.config }}\n      docs-only: ${{ steps.changes.outputs.docs }}\n      shard-count: ${{ steps.shards.outputs.count }}\n    steps:\n      - uses: actions\u002Fcheckout@v4\n        with:\n          fetch-depth: 0\n\n      - name: Detect changed file categories\n        id: changes\n        uses: dorny\u002Fpaths-filter@v3\n        with:\n          filters: |\n            backend:\n              - 'src\u002Fapi\u002F**'\n              - 'src\u002Fdb\u002F**'\n              - 'src\u002Flib\u002F**'\n            frontend:\n              - 'src\u002Fcomponents\u002F**'\n              - 'src\u002Fpages\u002F**'\n              - 'src\u002Fstyles\u002F**'\n            config:\n              - 'package.json'\n              - 'pnpm-lock.yaml'\n              - '*.config.*'\n            docs:\n              - '**\u002F*.md'\n              - 'docs\u002F**'\n\n      - name: Determine optimal shard count\n        id: shards\n        run: |\n          if [[ \"${{ steps.changes.outputs.config }}\" == \"true\" ]]; then\n            echo \"count=4\" >> $GITHUB_OUTPUT\n          elif [[ \"${{ steps.changes.outputs.backend }}\" == \"true\" && \"${{ steps.changes.outputs.frontend }}\" == \"true\" ]]; then\n            echo \"count=4\" >> $GITHUB_OUTPUT\n          elif [[ \"${{ steps.changes.outputs.backend }}\" == \"true\" || \"${{ steps.changes.outputs.frontend }}\" == \"true\" ]]; then\n            echo \"count=2\" >> $GITHUB_OUTPUT\n          else\n            echo \"count=1\" >> $GITHUB_OUTPUT\n          fi\n\n  # Step 2: Lint and type-check (fastest feedback)\n  lint:\n    runs-on: ubuntu-latest\n    needs: detect-changes\n    if: needs.detect-changes.outputs.docs-only != 'true'\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n          cache: 'pnpm'\n\n      - run: pnpm install --frozen-lockfile\n\n      - name: Run lint\n        run: pnpm lint\n\n      - name: Run type check\n        run: pnpm tsc --noEmit\n\n  # Step 3: Unit tests (fast, parallel shards)\n  unit-tests:\n    runs-on: ubuntu-latest\n    needs: [detect-changes, lint]\n    if: needs.detect-changes.outputs.docs-only != 'true'\n    strategy:\n      fail-fast: false\n      matrix:\n        shard: [1, 2]\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n          cache: 'pnpm'\n\n      - run: pnpm install --frozen-lockfile\n\n      - name: Restore test timings\n        uses: actions\u002Fcache@v4\n        with:\n          path: test-timings.json\n          key: test-timings-${{ github.ref }}\n          restore-keys: |\n            test-timings-refs\u002Fheads\u002Fmain\n            test-timings-\n\n      - name: Run unit tests (shard ${{ matrix.shard }}\u002F2)\n        run: |\n          pnpm vitest run \\\n            --reporter=json \\\n            --outputFile=vitest-results.json \\\n            --shard=${{ matrix.shard }}\u002F2\n\n      - name: Collect test timings\n        if: always()\n        run: npx tsx ci-config\u002Fscripts\u002Ftiming-collector.ts\n\n      - name: Save test timings\n        if: always()\n        uses: actions\u002Fcache\u002Fsave@v4\n        with:\n          path: test-timings.json\n          key: test-timings-${{ github.ref }}-${{ github.run_id }}-shard-${{ matrix.shard }}\n\n      - name: Upload test results\n        if: always()\n        uses: actions\u002Fupload-artifact@v4\n        with:\n          name: unit-results-shard-${{ matrix.shard }}\n          path: vitest-results.json\n          retention-days: 7\n\n  # Step 4: Integration tests (medium speed, conditional)\n  integration-tests:\n    runs-on: ubuntu-latest\n    needs: [detect-changes, lint]\n    if: |\n      needs.detect-changes.outputs.backend == 'true' ||\n      needs.detect-changes.outputs.config == 'true'\n    services:\n      postgres:\n        image: postgres:16\n        env:\n          POSTGRES_PASSWORD: test\n          POSTGRES_DB: test\n        ports:\n          - 5432:5432\n        options: >-\n          --health-cmd pg_isready\n          --health-interval 10s\n          --health-timeout 5s\n          --health-retries 5\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n          cache: 'pnpm'\n\n      - run: pnpm install --frozen-lockfile\n\n      - name: Run integration tests\n        run: pnpm run test:integration\n        env:\n          DATABASE_URL: postgresql:\u002F\u002Fpostgres:test@localhost:5432\u002Ftest\n\n  # Step 5: E2E tests (slowest, most shards, conditional)\n  e2e-tests:\n    runs-on: ubuntu-latest\n    needs: [detect-changes, lint]\n    if: |\n      needs.detect-changes.outputs.frontend == 'true' ||\n      needs.detect-changes.outputs.config == 'true'\n    strategy:\n      fail-fast: false\n      matrix:\n        shard: [1, 2, 3, 4]\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n          cache: 'pnpm'\n\n      - run: pnpm install --frozen-lockfile\n\n      - name: Cache Playwright browsers\n        uses: actions\u002Fcache@v4\n        with:\n          path: ~\u002F.cache\u002Fms-playwright\n          key: playwright-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}\n\n      - name: Install Playwright browsers\n        run: npx playwright install --with-deps chromium\n\n      - name: Build application\n        run: pnpm build\n\n      - name: Run E2E tests (shard ${{ matrix.shard }}\u002F4)\n        run: npx playwright test --shard=${{ matrix.shard }}\u002F4\n\n      - name: Upload failure artifacts\n        if: failure()\n        uses: actions\u002Fupload-artifact@v4\n        with:\n          name: e2e-results-shard-${{ matrix.shard }}\n          path: test-results\u002F\n          retention-days: 7\n```\n\n### Cache Warmup Workflow\n\nPre-populate caches on a schedule so the first PR of the day gets warm caches:\n\n```yaml\n# .github\u002Fworkflows\u002Fcache-warmup.yml\nname: Cache Warmup\n\non:\n  schedule:\n    - cron: '0 6 * * 1-5'  # Weekdays at 6 AM UTC\n  workflow_dispatch:\n\njobs:\n  warmup:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n\n      - name: Install dependencies\n        run: pnpm install --frozen-lockfile\n\n      - name: Cache node_modules\n        uses: actions\u002Fcache\u002Fsave@v4\n        with:\n          path: |\n            node_modules\n            ~\u002F.pnpm-store\n          key: pnpm-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}\n\n      - name: Install Playwright browsers\n        run: npx playwright install --with-deps chromium\n\n      - name: Cache Playwright\n        uses: actions\u002Fcache\u002Fsave@v4\n        with:\n          path: ~\u002F.cache\u002Fms-playwright\n          key: playwright-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}\n\n      - name: Build application\n        run: pnpm build\n\n      - name: Cache build artifacts\n        uses: actions\u002Fcache\u002Fsave@v4\n        with:\n          path: |\n            .next\u002Fcache\n            dist\n            .turbo\n          key: build-${{ runner.os }}-${{ hashFiles('tsconfig.json') }}-${{ github.sha }}\n```\n\n## Pipeline Analysis and Monitoring\n\n### Pipeline Duration Analyzer\n\nTrack and analyze pipeline performance over time to identify regressions and optimization opportunities:\n\n```typescript\n\u002F\u002F scripts\u002Fpipeline-analyzer.ts\n\ninterface PipelineRun {\n  runId: string;\n  branch: string;\n  timestamp: string;\n  totalDuration: number;\n  jobs: Array\u003C{\n    name: string;\n    duration: number;\n    status: 'success' | 'failure' | 'cancelled';\n    steps: Array\u003C{\n      name: string;\n      duration: number;\n    }>;\n  }>;\n}\n\ninterface PipelineAnalysis {\n  averageDuration: number;\n  medianDuration: number;\n  p95Duration: number;\n  slowestJobs: Array\u003C{ name: string; avgDuration: number }>;\n  slowestSteps: Array\u003C{ jobName: string; stepName: string; avgDuration: number }>;\n  cacheSavings: number;\n  recommendations: string[];\n}\n\nexport function analyzePipeline(runs: PipelineRun[]): PipelineAnalysis {\n  if (runs.length === 0) {\n    return {\n      averageDuration: 0,\n      medianDuration: 0,\n      p95Duration: 0,\n      slowestJobs: [],\n      slowestSteps: [],\n      cacheSavings: 0,\n      recommendations: [],\n    };\n  }\n\n  const durations = runs.map((r) => r.totalDuration).sort((a, b) => a - b);\n  const average = durations.reduce((a, b) => a + b, 0) \u002F durations.length;\n  const median = durations[Math.floor(durations.length \u002F 2)];\n  const p95 = durations[Math.floor(durations.length * 0.95)];\n\n  \u002F\u002F Aggregate job durations across runs\n  const jobDurations = new Map\u003Cstring, number[]>();\n  const stepDurations = new Map\u003Cstring, number[]>();\n\n  for (const run of runs) {\n    for (const job of run.jobs) {\n      if (!jobDurations.has(job.name)) {\n        jobDurations.set(job.name, []);\n      }\n      jobDurations.get(job.name)!.push(job.duration);\n\n      for (const step of job.steps) {\n        const key = `${job.name}::${step.name}`;\n        if (!stepDurations.has(key)) {\n          stepDurations.set(key, []);\n        }\n        stepDurations.get(key)!.push(step.duration);\n      }\n    }\n  }\n\n  const slowestJobs = Array.from(jobDurations.entries())\n    .map(([name, times]) => ({\n      name,\n      avgDuration: times.reduce((a, b) => a + b, 0) \u002F times.length,\n    }))\n    .sort((a, b) => b.avgDuration - a.avgDuration)\n    .slice(0, 5);\n\n  const slowestSteps = Array.from(stepDurations.entries())\n    .map(([key, times]) => {\n      const [jobName, stepName] = key.split('::');\n      return {\n        jobName,\n        stepName,\n        avgDuration: times.reduce((a, b) => a + b, 0) \u002F times.length,\n      };\n    })\n    .sort((a, b) => b.avgDuration - a.avgDuration)\n    .slice(0, 10);\n\n  const recommendations = generateOptimizationRecommendations(\n    average,\n    slowestJobs,\n    slowestSteps\n  );\n\n  return {\n    averageDuration: Math.round(average),\n    medianDuration: Math.round(median),\n    p95Duration: Math.round(p95),\n    slowestJobs,\n    slowestSteps,\n    cacheSavings: estimateCacheSavings(stepDurations),\n    recommendations,\n  };\n}\n\nfunction generateOptimizationRecommendations(\n  avgDuration: number,\n  slowestJobs: Array\u003C{ name: string; avgDuration: number }>,\n  slowestSteps: Array\u003C{ jobName: string; stepName: string; avgDuration: number }>\n): string[] {\n  const recommendations: string[] = [];\n\n  if (avgDuration > 600000) {\n    recommendations.push(\n      'Pipeline average exceeds 10 minutes. Consider increasing parallelization or enabling selective test execution.'\n    );\n  }\n\n  for (const job of slowestJobs) {\n    if (job.avgDuration > 300000) {\n      recommendations.push(\n        `Job \"${job.name}\" averages ${(job.avgDuration \u002F 60000).toFixed(1)}min. Consider splitting into shards.`\n      );\n    }\n  }\n\n  for (const step of slowestSteps) {\n    if (step.stepName.toLowerCase().includes('install') && step.avgDuration > 60000) {\n      recommendations.push(\n        `Step \"${step.stepName}\" in \"${step.jobName}\" takes ${(step.avgDuration \u002F 1000).toFixed(0)}s. Verify dependency caching is working correctly.`\n      );\n    }\n    if (step.stepName.toLowerCase().includes('build') && step.avgDuration > 120000) {\n      recommendations.push(\n        `Build step in \"${step.jobName}\" takes ${(step.avgDuration \u002F 1000).toFixed(0)}s. Consider incremental builds or build caching with Turborepo.`\n      );\n    }\n  }\n\n  return recommendations;\n}\n\nfunction estimateCacheSavings(stepDurations: Map\u003Cstring, number[]>): number {\n  let savings = 0;\n\n  for (const [key, times] of stepDurations) {\n    if (key.toLowerCase().includes('install') || key.toLowerCase().includes('cache')) {\n      if (times.length > 1) {\n        const first = times[0];\n        const subsequent = times.slice(1);\n        const avgSubsequent =\n          subsequent.reduce((a, b) => a + b, 0) \u002F subsequent.length;\n        if (first > avgSubsequent * 2) {\n          savings += (first - avgSubsequent) * subsequent.length;\n        }\n      }\n    }\n  }\n\n  return Math.round(savings);\n}\n```\n\n## Critical Path Optimization\n\n### Understanding the Pipeline Dependency Graph\n\nThe critical path is the longest sequential chain of jobs in your pipeline. Parallelizing jobs off the critical path does not reduce total duration. Focus optimization on the critical path:\n\n```typescript\n\u002F\u002F scripts\u002Fcritical-path.ts\n\ninterface JobNode {\n  name: string;\n  estimatedDuration: number;\n  dependencies: string[];\n  canParallelize: boolean;\n  shardable: boolean;\n}\n\nexport function findCriticalPath(jobs: JobNode[]): {\n  path: JobNode[];\n  duration: number;\n  parallelizableSavings: number;\n} {\n  const jobMap = new Map(jobs.map((j) => [j.name, j]));\n  const memo = new Map\u003Cstring, number>();\n\n  function longestPath(jobName: string): number {\n    if (memo.has(jobName)) return memo.get(jobName)!;\n\n    const job = jobMap.get(jobName);\n    if (!job) return 0;\n\n    let maxDepDuration = 0;\n    for (const dep of job.dependencies) {\n      maxDepDuration = Math.max(maxDepDuration, longestPath(dep));\n    }\n\n    const total = maxDepDuration + job.estimatedDuration;\n    memo.set(jobName, total);\n    return total;\n  }\n\n  \u002F\u002F Find the endpoint of the critical path\n  let maxDuration = 0;\n  let criticalEndJob = '';\n\n  for (const job of jobs) {\n    const duration = longestPath(job.name);\n    if (duration > maxDuration) {\n      maxDuration = duration;\n      criticalEndJob = job.name;\n    }\n  }\n\n  \u002F\u002F Reconstruct the critical path by walking backwards\n  const path: JobNode[] = [];\n  let current = criticalEndJob;\n\n  while (current) {\n    const job = jobMap.get(current);\n    if (!job) break;\n    path.unshift(job);\n\n    let nextJob = '';\n    let nextDuration = 0;\n    for (const dep of job.dependencies) {\n      const depDuration = memo.get(dep) || 0;\n      if (depDuration > nextDuration) {\n        nextDuration = depDuration;\n        nextJob = dep;\n      }\n    }\n    current = nextJob;\n  }\n\n  const totalSequentialDuration = jobs.reduce(\n    (sum, job) => sum + job.estimatedDuration,\n    0\n  );\n  const criticalPathDuration = path.reduce(\n    (sum, job) => sum + job.estimatedDuration,\n    0\n  );\n\n  return {\n    path,\n    duration: criticalPathDuration,\n    parallelizableSavings: totalSequentialDuration - criticalPathDuration,\n  };\n}\n```\n\n## Best Practices\n\n1. **Measure before optimizing.** Profile your current pipeline to identify actual bottlenecks. The slowest step may not be what you expect. Use pipeline analytics to find the real performance killers before implementing any optimizations.\n\n2. **Use concurrency controls to cancel redundant runs.** When a developer pushes multiple commits in quick succession, cancel the in-progress run for the previous commit. The GitHub Actions `concurrency` key with `cancel-in-progress: true` handles this automatically.\n\n3. **Cache aggressively but validate cache correctness.** Cache node_modules, build artifacts, Playwright browsers, and Docker layers. But always verify that stale caches do not cause false-positive test results. Include lock file hashes in cache keys to invalidate on dependency changes.\n\n4. **Split E2E tests across more shards than unit tests.** E2E tests are typically 10-50x slower than unit tests. A suite that benefits from 2 unit test shards may need 4-8 E2E test shards for balanced execution times.\n\n5. **Run lint and type-check before tests.** These checks are fast (typically under 60 seconds) and catch a large class of issues. Running them before expensive test suites provides faster initial feedback when they fail.\n\n6. **Use fail-fast: false for sharded jobs.** When tests are split across shards, a failure in shard 1 should not cancel shard 3. You want complete results from all shards to see the full scope of failures.\n\n7. **Collect and store test timing data persistently.** Duration-based test splitting requires historical data to be effective. Collect timings after every CI run and cache them for future runs. Use exponential moving averages to smooth outliers.\n\n8. **Implement selective test execution with a safety net.** When change detection cannot determine which tests to run (e.g., for config file changes), default to running all tests. A false positive (running unnecessary tests) is always safer than a false negative (missing a regression).\n\n9. **Pre-warm caches on a schedule.** Run a scheduled workflow that installs dependencies and populates caches before the workday begins. This ensures the first PR of the day gets a cache hit instead of a cold start.\n\n10. **Monitor pipeline duration trends over time.** Track P50 and P95 pipeline durations weekly. A gradual increase of 30 seconds per week adds up to 25 minutes over a year. Catch regressions early with automated alerting on duration spikes.\n\n11. **Use service containers for integration test dependencies.** Instead of installing PostgreSQL or Redis in your workflow steps, use GitHub Actions service containers. They start in parallel with job setup and are ready when your tests need them.\n\n12. **Keep Docker images minimal for CI runners.** If your CI uses custom Docker images, minimize their size. Every megabyte of image that needs to be pulled adds to every pipeline run. Use multi-stage builds and slim base images.\n\n## Anti-Patterns to Avoid\n\n**Running all tests on every PR regardless of changes.** A documentation-only PR does not need E2E tests. A frontend-only change does not need backend integration tests. Implement change detection to skip unnecessary work and save both time and compute resources.\n\n**Using sequential job execution when parallelism is possible.** If unit tests and E2E tests have no dependency on each other, run them in parallel. Restructure your pipeline so that only actual data dependencies create sequential chains.\n\n**Caching too broadly or too narrowly.** Caching `node_modules` without including the lock file hash in the cache key means stale dependencies persist. Caching only `node_modules` but not Playwright browsers means downloading 100+ MB of browser binaries on every run.\n\n**Splitting tests by file count instead of duration.** Ten test files may take 10 seconds or 10 minutes depending on their content. Always use duration-based splitting when timing data is available to ensure balanced shard execution times.\n\n**Ignoring the critical path.** Parallelizing jobs that are already off the critical path does not reduce total pipeline duration. Identify and focus optimization on the jobs that form the longest sequential chain.\n\n**Not canceling redundant pipeline runs.** Without concurrency controls, every push creates a new pipeline run. Three pushes in 5 minutes create three full runs, wasting resources and delaying results for other developers waiting in the queue.\n\n**Hard-coding shard counts.** A static 4-shard configuration may be wasteful for small PRs and insufficient for full test runs on main. Dynamically determine shard counts based on the scope of changes detected.\n\n**Skipping tests to speed up the pipeline.** Disabling tests is not optimization; it is risk accumulation. Optimize the tests themselves (reduce flakiness, parallelize, cache) rather than removing them from the pipeline.\n\n## Debugging Tips\n\n**Cache misses on every run.** Verify that the cache key includes only deterministic inputs. Environment variables, timestamps, and random values in cache keys cause every run to miss. Use `hashFiles()` on lock files and configuration files for consistent keys.\n\n**Sharded tests produce inconsistent results.** Test isolation issues become visible under sharding because tests run in different orders and different processes. Identify tests that depend on shared state (global variables, database records, file system artifacts) and fix them to be truly independent.\n\n**Selective test execution misses a regression.** Review your change-to-test mapping configuration. Common gaps include: shared utility files that are not mapped to all their consumers, configuration changes that should trigger full runs, and transitive dependencies that are not tracked in the change map.\n\n**Pipeline duration increases after adding caching.** This can happen when cache save and restore steps take longer than the operation they are caching. Very small caches (under 10 MB) may not be worth the overhead. Profile the cache steps themselves to verify they provide net savings.\n\n**Test timings file grows unbounded.** Prune entries for test files that no longer exist. When files are renamed or deleted, their timing entries remain in the file indefinitely. Add a cleanup step that removes entries for files not present in the current codebase.\n\n**GitHub Actions job matrix generates too many combinations.** Matrix strategies with multiple dimensions (OS x Node version x shard) can produce dozens of jobs. Use `include` and `exclude` to limit combinations to only those that provide unique value.\n\n**Jobs fail with \"No space left on device.\"** CI runners have limited disk space. Large caches, Docker images, and build artifacts can exhaust the available storage. Add cleanup steps between resource-intensive operations, use slimmer base images, and prune unnecessary artifacts before running tests.\n","# CI Pipeline Optimizer\n\nSlow CI pipelines are one of the most common productivity killers in software teams. A test suite that takes 30 minutes to run means developers context-switch away from their work, batch multiple changes into fewer PRs to avoid waiting, and eventually start skipping CI altogether. This skill addresses CI pipeline performance through four complementary strategies: intelligent test splitting across parallel workers, selective test execution based on code changes, aggressive caching of dependencies and build artifacts, and pipeline architecture that minimizes total wall-clock time. The techniques apply to any CI system but use GitHub Actions as the primary example, with patterns that transfer to GitLab CI, CircleCI, Jenkins, and other platforms.\n\n## Core Principles\n\n### 1. Wall-Clock Time Is the Only Metric That Matters\n\nCPU time, billable minutes, and total test count are secondary metrics. The developer waits for wall-clock time. A pipeline that uses 60 CPU-minutes across 10 parallel workers in 6 minutes is vastly preferable to a pipeline that uses 20 CPU-minutes sequentially in 20 minutes. Optimize for the duration the developer experiences.\n\n### 2. Never Run Tests You Do Not Need\n\nIf a PR changes only documentation files, running the full test suite is waste. If a PR modifies only the frontend, running backend integration tests is waste. Selective test execution identifies the minimum set of tests needed to validate a specific change, with a safety net that defaults to running everything when the analysis is uncertain.\n\n### 3. Cache Everything That Does Not Change\n\nDependencies, build artifacts, Docker layers, and browser binaries are the same across most CI runs. Downloading and building them from scratch on every run is unnecessary. Aggressive caching can eliminate minutes from every pipeline execution.\n\n### 4. Split Tests by Duration, Not by File Count\n\nNaive test splitting distributes files evenly across workers. But if one file contains a 5-minute test and another contains a 5-second test, the distribution is severely unbalanced. Intelligent splitting uses historical duration data to distribute tests so that all workers finish at approximately the same time.\n\n### 5. Fail Fast, Report Completely\n\nRun the fastest tests first. If unit tests catch a bug in 30 seconds, there is no reason to wait 10 minutes for e2e tests to also catch it. Structure the pipeline so the cheapest checks (lint, type-check) run first to provide the fastest possible feedback, but still collect complete results from all shards for thorough reporting.\n\n## Project Structure\n\n```\nci-config\u002F\n  scripts\u002F\n    split-tests.ts\n    detect-changes.ts\n    cache-manager.ts\n    timing-collector.ts\n    pipeline-analyzer.ts\n  config\u002F\n    test-groups.json\n    change-map.json\n    cache-config.json\n  .github\u002F\n    workflows\u002F\n      ci-optimized.yml\n      ci-selective.yml\n      cache-warmup.yml\n  package.json\n  tsconfig.json\n```\n\nThe `scripts\u002F` directory contains automation tools for test splitting, change detection, cache management, and pipeline analysis. The `config\u002F` directory holds the mapping from file changes to test groups, cache key definitions, and test group specifications.\n\n## Test Splitting Strategies\n\n### Duration-Based Test Splitting\n\nSplit tests across parallel workers based on historical execution times so that each worker runs for approximately the same duration:\n\n```typescript\n\u002F\u002F scripts\u002Fsplit-tests.ts\n\nimport { readFileSync, writeFileSync, existsSync } from 'fs';\nimport { execSync } from 'child_process';\n\ninterface TestTiming {\n  file: string;\n  duration: number;\n  lastRun: string;\n}\n\ninterface SplitResult {\n  shardIndex: number;\n  files: string[];\n  estimatedDuration: number;\n}\n\nexport function splitTestsByDuration(\n  testFiles: string[],\n  shardCount: number,\n  timingsFile: string = 'test-timings.json'\n): SplitResult[] {\n  \u002F\u002F Load historical timings\n  const timings = loadTimings(timingsFile);\n\n  \u002F\u002F Create a list of files with their estimated duration\n  const fileTimings: Array\u003C{ file: string; duration: number }> = testFiles.map((file) => ({\n    file,\n    duration: timings.get(file) || estimateDefaultDuration(file),\n  }));\n\n  \u002F\u002F Sort by duration descending (greedy algorithm: assign largest jobs first)\n  fileTimings.sort((a, b) => b.duration - a.duration);\n\n  \u002F\u002F Initialize shards\n  const shards: SplitResult[] = Array.from({ length: shardCount }, (_, i) => ({\n    shardIndex: i,\n    files: [],\n    estimatedDuration: 0,\n  }));\n\n  \u002F\u002F Greedy assignment: always assign to the shard with the least total duration\n  for (const fileTiming of fileTimings) {\n    const lightest = shards.reduce((min, shard) =>\n      shard.estimatedDuration \u003C min.estimatedDuration ? shard : min\n    );\n    lightest.files.push(fileTiming.file);\n    lightest.estimatedDuration += fileTiming.duration;\n  }\n\n  return shards;\n}\n\nfunction loadTimings(timingsFile: string): Map\u003Cstring, number> {\n  const timings = new Map\u003Cstring, number>();\n\n  if (!existsSync(timingsFile)) {\n    return timings;\n  }\n\n  try {\n    const data: TestTiming[] = JSON.parse(readFileSync(timingsFile, 'utf-8'));\n    for (const entry of data) {\n      timings.set(entry.file, entry.duration);\n    }\n  } catch {\n    console.warn(`Failed to parse timings file: ${timingsFile}`);\n  }\n\n  return timings;\n}\n\nfunction estimateDefaultDuration(file: string): number {\n  \u002F\u002F Heuristic estimates based on test type when no historical data exists\n  if (file.includes('.e2e.') || file.includes('e2e\u002F')) return 30000;\n  if (file.includes('.integration.') || file.includes('integration\u002F')) return 10000;\n  if (file.includes('.spec.')) return 5000;\n  return 3000; \u002F\u002F Default estimate for unit tests\n}\n\n\u002F\u002F CLI entry point for use in GitHub Actions\nfunction main(): void {\n  const shardIndex = parseInt(process.env.SHARD_INDEX || '0', 10);\n  const shardCount = parseInt(process.env.SHARD_COUNT || '1', 10);\n\n  \u002F\u002F Discover all test files\n  const testFilesOutput = execSync(\n    'find . -name \"*.test.ts\" -o -name \"*.spec.ts\" | grep -v node_modules',\n    { encoding: 'utf-8' }\n  );\n  const testFiles = testFilesOutput.trim().split('\\n').filter(Boolean);\n\n  const shards = splitTestsByDuration(testFiles, shardCount);\n  const myShard = shards[shardIndex];\n\n  if (!myShard) {\n    console.error(`Invalid shard index ${shardIndex} for ${shardCount} shards`);\n    process.exit(1);\n  }\n\n  console.log(\n    `Shard ${shardIndex + 1}\u002F${shardCount}: ${myShard.files.length} files, ~${(myShard.estimatedDuration \u002F 1000).toFixed(1)}s`\n  );\n\n  \u002F\u002F Write shard files list for the test runner to consume\n  writeFileSync('shard-files.txt', myShard.files.join('\\n'), 'utf-8');\n}\n\nmain();\n```\n\n### Collecting Test Timings\n\nAfter each CI run, collect and store test execution timings for future split optimization:\n\n```typescript\n\u002F\u002F scripts\u002Ftiming-collector.ts\n\nimport { readFileSync, writeFileSync, existsSync } from 'fs';\n\ninterface JestResult {\n  testResults: Array\u003C{\n    testFilePath: string;\n    perfStats: {\n      runtime: number;\n    };\n  }>;\n}\n\ninterface PlaywrightResult {\n  suites: Array\u003C{\n    file: string;\n    specs: Array\u003C{\n      tests: Array\u003C{\n        results: Array\u003C{\n          duration: number;\n        }>;\n      }>;\n    }>;\n  }>;\n}\n\ninterface TestTiming {\n  file: string;\n  duration: number;\n  lastRun: string;\n}\n\nexport function collectJestTimings(resultsFile: string): TestTiming[] {\n  const results: JestResult = JSON.parse(readFileSync(resultsFile, 'utf-8'));\n\n  return results.testResults.map((test) => ({\n    file: test.testFilePath.replace(process.cwd() + '\u002F', ''),\n    duration: test.perfStats.runtime,\n    lastRun: new Date().toISOString(),\n  }));\n}\n\nexport function collectPlaywrightTimings(resultsFile: string): TestTiming[] {\n  const results: PlaywrightResult = JSON.parse(readFileSync(resultsFile, 'utf-8'));\n  const timings: TestTiming[] = [];\n\n  for (const suite of results.suites) {\n    let totalDuration = 0;\n    for (const spec of suite.specs) {\n      for (const test of spec.tests) {\n        for (const result of test.results) {\n          totalDuration += result.duration;\n        }\n      }\n    }\n\n    timings.push({\n      file: suite.file,\n      duration: totalDuration,\n      lastRun: new Date().toISOString(),\n    });\n  }\n\n  return timings;\n}\n\nexport function mergeTimings(\n  existing: TestTiming[],\n  latest: TestTiming[]\n): TestTiming[] {\n  const map = new Map\u003Cstring, TestTiming>();\n\n  for (const timing of existing) {\n    map.set(timing.file, timing);\n  }\n\n  \u002F\u002F Merge with exponential moving average to smooth out outliers\n  for (const timing of latest) {\n    const prev = map.get(timing.file);\n    if (prev) {\n      \u002F\u002F EMA with alpha = 0.3 gives 70% weight to history, 30% to new data\n      const smoothedDuration = prev.duration * 0.7 + timing.duration * 0.3;\n      map.set(timing.file, {\n        file: timing.file,\n        duration: Math.round(smoothedDuration),\n        lastRun: timing.lastRun,\n      });\n    } else {\n      map.set(timing.file, timing);\n    }\n  }\n\n  return Array.from(map.values());\n}\n\nfunction main(): void {\n  const timingsFile = 'test-timings.json';\n  const existing: TestTiming[] = existsSync(timingsFile)\n    ? JSON.parse(readFileSync(timingsFile, 'utf-8'))\n    : [];\n\n  let latest: TestTiming[] = [];\n\n  if (existsSync('jest-results.json')) {\n    latest = collectJestTimings('jest-results.json');\n  } else if (existsSync('playwright-results.json')) {\n    latest = collectPlaywrightTimings('playwright-results.json');\n  }\n\n  if (latest.length > 0) {\n    const merged = mergeTimings(existing, latest);\n    writeFileSync(timingsFile, JSON.stringify(merged, null, 2), 'utf-8');\n    console.log(`Updated timings for ${latest.length} test files (${merged.length} total)`);\n  } else {\n    console.warn('No test results found to collect timings from');\n  }\n}\n\nmain();\n```\n\n## Selective Test Execution\n\n### Change Detection Engine\n\nDetect which files changed and determine which test groups need to run:\n\n```typescript\n\u002F\u002F scripts\u002Fdetect-changes.ts\n\nimport { execSync } from 'child_process';\nimport { readFileSync, existsSync } from 'fs';\n\ninterface ChangeMap {\n  patterns: Array\u003C{\n    glob: string;\n    testGroups: string[];\n    description: string;\n  }>;\n  testGroups: Record\u003C\n    string,\n    {\n      command: string;\n      files?: string[];\n      description: string;\n    }\n  >;\n}\n\ninterface DetectedChanges {\n  changedFiles: string[];\n  testGroupsToRun: Set\u003Cstring>;\n  skipReason?: string;\n}\n\nexport function detectChanges(baseBranch: string = 'main'): DetectedChanges {\n  let changedFiles: string[];\n\n  try {\n    const diffOutput = execSync(\n      `git diff --name-only origin\u002F${baseBranch}...HEAD`,\n      { encoding: 'utf-8' }\n    );\n    changedFiles = diffOutput.trim().split('\\n').filter(Boolean);\n  } catch {\n    const diffOutput = execSync('git diff --name-only HEAD~1', {\n      encoding: 'utf-8',\n    });\n    changedFiles = diffOutput.trim().split('\\n').filter(Boolean);\n  }\n\n  if (changedFiles.length === 0) {\n    return {\n      changedFiles: [],\n      testGroupsToRun: new Set(),\n      skipReason: 'No files changed',\n    };\n  }\n\n  const changeMap = loadChangeMap();\n  const testGroups = new Set\u003Cstring>();\n\n  for (const file of changedFiles) {\n    for (const pattern of changeMap.patterns) {\n      if (matchGlob(file, pattern.glob)) {\n        for (const group of pattern.testGroups) {\n          testGroups.add(group);\n        }\n      }\n    }\n  }\n\n  \u002F\u002F Safety net: if no patterns matched, run all tests\n  if (testGroups.size === 0) {\n    testGroups.add('all');\n  }\n\n  return {\n    changedFiles,\n    testGroupsToRun: testGroups,\n  };\n}\n\nfunction loadChangeMap(): ChangeMap {\n  const configPath = 'ci-config\u002Fconfig\u002Fchange-map.json';\n  if (!existsSync(configPath)) {\n    return getDefaultChangeMap();\n  }\n  return JSON.parse(readFileSync(configPath, 'utf-8'));\n}\n\nfunction getDefaultChangeMap(): ChangeMap {\n  return {\n    patterns: [\n      {\n        glob: 'src\u002Fapi\u002F**',\n        testGroups: ['unit', 'api-integration'],\n        description: 'API source changes trigger unit and integration tests',\n      },\n      {\n        glob: 'src\u002Fcomponents\u002F**',\n        testGroups: ['unit', 'component'],\n        description: 'UI component changes trigger unit and component tests',\n      },\n      {\n        glob: 'src\u002Fpages\u002F**',\n        testGroups: ['unit', 'e2e'],\n        description: 'Page-level changes trigger unit and E2E tests',\n      },\n      {\n        glob: 'src\u002Flib\u002F**',\n        testGroups: ['unit'],\n        description: 'Library changes trigger unit tests',\n      },\n      {\n        glob: 'src\u002Fdb\u002F**',\n        testGroups: ['unit', 'api-integration', 'e2e'],\n        description: 'Database changes trigger all test types',\n      },\n      {\n        glob: 'package.json',\n        testGroups: ['all'],\n        description: 'Dependency changes require full test run',\n      },\n      {\n        glob: '*.config.*',\n        testGroups: ['all'],\n        description: 'Config changes require full test run',\n      },\n      {\n        glob: '**\u002F*.md',\n        testGroups: ['docs-only'],\n        description: 'Documentation-only changes skip tests',\n      },\n      {\n        glob: '.github\u002F**',\n        testGroups: ['ci-only'],\n        description: 'CI configuration changes need validation',\n      },\n    ],\n    testGroups: {\n      all: { command: 'npm test', description: 'Full test suite' },\n      unit: { command: 'npm run test:unit', description: 'Unit tests only' },\n      'api-integration': {\n        command: 'npm run test:integration',\n        description: 'API integration tests',\n      },\n      component: { command: 'npm run test:components', description: 'Component tests' },\n      e2e: { command: 'npx playwright test', description: 'End-to-end tests' },\n      'docs-only': {\n        command: 'echo \"No tests needed for docs-only changes\"',\n        description: 'Skip tests',\n      },\n      'ci-only': {\n        command: 'echo \"CI config changed - validate workflow syntax only\"',\n        description: 'Validate CI config',\n      },\n    },\n  };\n}\n\nfunction matchGlob(filePath: string, pattern: string): boolean {\n  const regexPattern = pattern\n    .replace(\u002F\\*\\*\u002Fg, '\u003C\u003C\u003CGLOBSTAR>>>')\n    .replace(\u002F\\*\u002Fg, '[^\u002F]*')\n    .replace(\u002F\u003C\u003C\u003CGLOBSTAR>>>\u002Fg, '.*')\n    .replace(\u002F\\?\u002Fg, '.');\n  return new RegExp(`^${regexPattern}$`).test(filePath);\n}\n\nfunction main(): void {\n  const baseBranch = process.env.BASE_BRANCH || 'main';\n  const result = detectChanges(baseBranch);\n\n  console.log(`Changed files: ${result.changedFiles.length}`);\n  console.log(`Test groups to run: ${Array.from(result.testGroupsToRun).join(', ')}`);\n\n  if (result.skipReason) {\n    console.log(`Skip reason: ${result.skipReason}`);\n  }\n\n  \u002F\u002F Output for GitHub Actions\n  const groups = Array.from(result.testGroupsToRun);\n  const shouldRun = groups.length > 0 && !groups.includes('docs-only');\n  console.log(`::set-output name=test-groups::${JSON.stringify(groups)}`);\n  console.log(`::set-output name=should-run-tests::${shouldRun}`);\n}\n\nmain();\n```\n\n## Caching Strategies\n\n### Multi-Layer Cache Configuration\n\n```typescript\n\u002F\u002F scripts\u002Fcache-manager.ts\n\nimport { readFileSync, existsSync } from 'fs';\nimport { createHash } from 'crypto';\n\ninterface CacheLayer {\n  name: string;\n  paths: string[];\n  keyFiles: string[];\n  fallbackKeys: string[];\n  maxAge: number;\n}\n\ninterface CacheConfig {\n  layers: CacheLayer[];\n}\n\nexport function generateCacheKeys(config: CacheConfig): Array\u003C{\n  name: string;\n  key: string;\n  restoreKeys: string[];\n  paths: string[];\n}> {\n  const platform = process.env.RUNNER_OS || process.platform;\n\n  return config.layers.map((layer) => {\n    const fileHashes = layer.keyFiles\n      .filter((f) => existsSync(f))\n      .map((f) => hashFile(f))\n      .join('-');\n\n    const key = `${platform}-${layer.name}-${fileHashes}`;\n\n    const restoreKeys = layer.fallbackKeys.map(\n      (fallback) => `${platform}-${layer.name}-${fallback}`\n    );\n\n    return {\n      name: layer.name,\n      key,\n      restoreKeys,\n      paths: layer.paths,\n    };\n  });\n}\n\nfunction hashFile(filePath: string): string {\n  const content = readFileSync(filePath);\n  return createHash('sha256').update(content).digest('hex').substring(0, 16);\n}\n\nexport const DEFAULT_CACHE_CONFIG: CacheConfig = {\n  layers: [\n    {\n      name: 'node-modules',\n      paths: ['node_modules', '~\u002F.pnpm-store'],\n      keyFiles: ['pnpm-lock.yaml', 'package.json'],\n      fallbackKeys: [''],\n      maxAge: 604800000, \u002F\u002F 7 days\n    },\n    {\n      name: 'playwright-browsers',\n      paths: ['~\u002F.cache\u002Fms-playwright'],\n      keyFiles: ['package.json'],\n      fallbackKeys: [''],\n      maxAge: 2592000000, \u002F\u002F 30 days\n    },\n    {\n      name: 'build-cache',\n      paths: ['.next\u002Fcache', 'dist', '.turbo'],\n      keyFiles: ['tsconfig.json', 'next.config.js'],\n      fallbackKeys: [''],\n      maxAge: 86400000, \u002F\u002F 1 day\n    },\n    {\n      name: 'test-timings',\n      paths: ['test-timings.json'],\n      keyFiles: [],\n      fallbackKeys: [''],\n      maxAge: 2592000000, \u002F\u002F 30 days\n    },\n  ],\n};\n```\n\n## GitHub Actions Optimized Pipeline\n\n### Full Optimized CI Workflow\n\n```yaml\n# .github\u002Fworkflows\u002Fci-optimized.yml\nname: Optimized CI\n\non:\n  push:\n    branches: [main]\n  pull_request:\n    branches: [main]\n\nconcurrency:\n  group: ci-${{ github.ref }}\n  cancel-in-progress: true\n\njobs:\n  # Step 1: Detect changes and determine what to test\n  detect-changes:\n    runs-on: ubuntu-latest\n    outputs:\n      backend: ${{ steps.changes.outputs.backend }}\n      frontend: ${{ steps.changes.outputs.frontend }}\n      config: ${{ steps.changes.outputs.config }}\n      docs-only: ${{ steps.changes.outputs.docs }}\n      shard-count: ${{ steps.shards.outputs.count }}\n    steps:\n      - uses: actions\u002Fcheckout@v4\n        with:\n          fetch-depth: 0\n\n      - name: Detect changed file categories\n        id: changes\n        uses: dorny\u002Fpaths-filter@v3\n        with:\n          filters: |\n            backend:\n              - 'src\u002Fapi\u002F**'\n              - 'src\u002Fdb\u002F**'\n              - 'src\u002Flib\u002F**'\n            frontend:\n              - 'src\u002Fcomponents\u002F**'\n              - 'src\u002Fpages\u002F**'\n              - 'src\u002Fstyles\u002F**'\n            config:\n              - 'package.json'\n              - 'pnpm-lock.yaml'\n              - '*.config.*'\n            docs:\n              - '**\u002F*.md'\n              - 'docs\u002F**'\n\n      - name: Determine optimal shard count\n        id: shards\n        run: |\n          if [[ \"${{ steps.changes.outputs.config }}\" == \"true\" ]]; then\n            echo \"count=4\" >> $GITHUB_OUTPUT\n          elif [[ \"${{ steps.changes.outputs.backend }}\" == \"true\" && \"${{ steps.changes.outputs.frontend }}\" == \"true\" ]]; then\n            echo \"count=4\" >> $GITHUB_OUTPUT\n          elif [[ \"${{ steps.changes.outputs.backend }}\" == \"true\" || \"${{ steps.changes.outputs.frontend }}\" == \"true\" ]]; then\n            echo \"count=2\" >> $GITHUB_OUTPUT\n          else\n            echo \"count=1\" >> $GITHUB_OUTPUT\n          fi\n\n  # Step 2: Lint and type-check (fastest feedback)\n  lint:\n    runs-on: ubuntu-latest\n    needs: detect-changes\n    if: needs.detect-changes.outputs.docs-only != 'true'\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n          cache: 'pnpm'\n\n      - run: pnpm install --frozen-lockfile\n\n      - name: Run lint\n        run: pnpm lint\n\n      - name: Run type check\n        run: pnpm tsc --noEmit\n\n  # Step 3: Unit tests (fast, parallel shards)\n  unit-tests:\n    runs-on: ubuntu-latest\n    needs: [detect-changes, lint]\n    if: needs.detect-changes.outputs.docs-only != 'true'\n    strategy:\n      fail-fast: false\n      matrix:\n        shard: [1, 2]\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n          cache: 'pnpm'\n\n      - run: pnpm install --frozen-lockfile\n\n      - name: Restore test timings\n        uses: actions\u002Fcache@v4\n        with:\n          path: test-timings.json\n          key: test-timings-${{ github.ref }}\n          restore-keys: |\n            test-timings-refs\u002Fheads\u002Fmain\n            test-timings-\n\n      - name: Run unit tests (shard ${{ matrix.shard }}\u002F2)\n        run: |\n          pnpm vitest run \\\n            --reporter=json \\\n            --outputFile=vitest-results.json \\\n            --shard=${{ matrix.shard }}\u002F2\n\n      - name: Collect test timings\n        if: always()\n        run: npx tsx ci-config\u002Fscripts\u002Ftiming-collector.ts\n\n      - name: Save test timings\n        if: always()\n        uses: actions\u002Fcache\u002Fsave@v4\n        with:\n          path: test-timings.json\n          key: test-timings-${{ github.ref }}-${{ github.run_id }}-shard-${{ matrix.shard }}\n\n      - name: Upload test results\n        if: always()\n        uses: actions\u002Fupload-artifact@v4\n        with:\n          name: unit-results-shard-${{ matrix.shard }}\n          path: vitest-results.json\n          retention-days: 7\n\n  # Step 4: Integration tests (medium speed, conditional)\n  integration-tests:\n    runs-on: ubuntu-latest\n    needs: [detect-changes, lint]\n    if: |\n      needs.detect-changes.outputs.backend == 'true' ||\n      needs.detect-changes.outputs.config == 'true'\n    services:\n      postgres:\n        image: postgres:16\n        env:\n          POSTGRES_PASSWORD: test\n          POSTGRES_DB: test\n        ports:\n          - 5432:5432\n        options: >-\n          --health-cmd pg_isready\n          --health-interval 10s\n          --health-timeout 5s\n          --health-retries 5\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n          cache: 'pnpm'\n\n      - run: pnpm install --frozen-lockfile\n\n      - name: Run integration tests\n        run: pnpm run test:integration\n        env:\n          DATABASE_URL: postgresql:\u002F\u002Fpostgres:test@localhost:5432\u002Ftest\n\n  # Step 5: E2E tests (slowest, most shards, conditional)\n  e2e-tests:\n    runs-on: ubuntu-latest\n    needs: [detect-changes, lint]\n    if: |\n      needs.detect-changes.outputs.frontend == 'true' ||\n      needs.detect-changes.outputs.config == 'true'\n    strategy:\n      fail-fast: false\n      matrix:\n        shard: [1, 2, 3, 4]\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n          cache: 'pnpm'\n\n      - run: pnpm install --frozen-lockfile\n\n      - name: Cache Playwright browsers\n        uses: actions\u002Fcache@v4\n        with:\n          path: ~\u002F.cache\u002Fms-playwright\n          key: playwright-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}\n\n      - name: Install Playwright browsers\n        run: npx playwright install --with-deps chromium\n\n      - name: Build application\n        run: pnpm build\n\n      - name: Run E2E tests (shard ${{ matrix.shard }}\u002F4)\n        run: npx playwright test --shard=${{ matrix.shard }}\u002F4\n\n      - name: Upload failure artifacts\n        if: failure()\n        uses: actions\u002Fupload-artifact@v4\n        with:\n          name: e2e-results-shard-${{ matrix.shard }}\n          path: test-results\u002F\n          retention-days: 7\n```\n\n### Cache Warmup Workflow\n\nPre-populate caches on a schedule so the first PR of the day gets warm caches:\n\n```yaml\n# .github\u002Fworkflows\u002Fcache-warmup.yml\nname: Cache Warmup\n\non:\n  schedule:\n    - cron: '0 6 * * 1-5'  # Weekdays at 6 AM UTC\n  workflow_dispatch:\n\njobs:\n  warmup:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions\u002Fcheckout@v4\n\n      - uses: pnpm\u002Faction-setup@v2\n        with:\n          version: 9\n\n      - uses: actions\u002Fsetup-node@v4\n        with:\n          node-version: 20\n\n      - name: Install dependencies\n        run: pnpm install --frozen-lockfile\n\n      - name: Cache node_modules\n        uses: actions\u002Fcache\u002Fsave@v4\n        with:\n          path: |\n            node_modules\n            ~\u002F.pnpm-store\n          key: pnpm-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}\n\n      - name: Install Playwright browsers\n        run: npx playwright install --with-deps chromium\n\n      - name: Cache Playwright\n        uses: actions\u002Fcache\u002Fsave@v4\n        with:\n          path: ~\u002F.cache\u002Fms-playwright\n          key: playwright-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}\n\n      - name: Build application\n        run: pnpm build\n\n      - name: Cache build artifacts\n        uses: actions\u002Fcache\u002Fsave@v4\n        with:\n          path: |\n            .next\u002Fcache\n            dist\n            .turbo\n          key: build-${{ runner.os }}-${{ hashFiles('tsconfig.json') }}-${{ github.sha }}\n```\n\n## Pipeline Analysis and Monitoring\n\n### Pipeline Duration Analyzer\n\nTrack and analyze pipeline performance over time to identify regressions and optimization opportunities:\n\n```typescript\n\u002F\u002F scripts\u002Fpipeline-analyzer.ts\n\ninterface PipelineRun {\n  runId: string;\n  branch: string;\n  timestamp: string;\n  totalDuration: number;\n  jobs: Array\u003C{\n    name: string;\n    duration: number;\n    status: 'success' | 'failure' | 'cancelled';\n    steps: Array\u003C{\n      name: string;\n      duration: number;\n    }>;\n  }>;\n}\n\ninterface PipelineAnalysis {\n  averageDuration: number;\n  medianDuration: number;\n  p95Duration: number;\n  slowestJobs: Array\u003C{ name: string; avgDuration: number }>;\n  slowestSteps: Array\u003C{ jobName: string; stepName: string; avgDuration: number }>;\n  cacheSavings: number;\n  recommendations: string[];\n}\n\nexport function analyzePipeline(runs: PipelineRun[]): PipelineAnalysis {\n  if (runs.length === 0) {\n    return {\n      averageDuration: 0,\n      medianDuration: 0,\n      p95Duration: 0,\n      slowestJobs: [],\n      slowestSteps: [],\n      cacheSavings: 0,\n      recommendations: [],\n    };\n  }\n\n  const durations = runs.map((r) => r.totalDuration).sort((a, b) => a - b);\n  const average = durations.reduce((a, b) => a + b, 0) \u002F durations.length;\n  const median = durations[Math.floor(durations.length \u002F 2)];\n  const p95 = durations[Math.floor(durations.length * 0.95)];\n\n  \u002F\u002F Aggregate job durations across runs\n  const jobDurations = new Map\u003Cstring, number[]>();\n  const stepDurations = new Map\u003Cstring, number[]>();\n\n  for (const run of runs) {\n    for (const job of run.jobs) {\n      if (!jobDurations.has(job.name)) {\n        jobDurations.set(job.name, []);\n      }\n      jobDurations.get(job.name)!.push(job.duration);\n\n      for (const step of job.steps) {\n        const key = `${job.name}::${step.name}`;\n        if (!stepDurations.has(key)) {\n          stepDurations.set(key, []);\n        }\n        stepDurations.get(key)!.push(step.duration);\n      }\n    }\n  }\n\n  const slowestJobs = Array.from(jobDurations.entries())\n    .map(([name, times]) => ({\n      name,\n      avgDuration: times.reduce((a, b) => a + b, 0) \u002F times.length,\n    }))\n    .sort((a, b) => b.avgDuration - a.avgDuration)\n    .slice(0, 5);\n\n  const slowestSteps = Array.from(stepDurations.entries())\n    .map(([key, times]) => {\n      const [jobName, stepName] = key.split('::');\n      return {\n        jobName,\n        stepName,\n        avgDuration: times.reduce((a, b) => a + b, 0) \u002F times.length,\n      };\n    })\n    .sort((a, b) => b.avgDuration - a.avgDuration)\n    .slice(0, 10);\n\n  const recommendations = generateOptimizationRecommendations(\n    average,\n    slowestJobs,\n    slowestSteps\n  );\n\n  return {\n    averageDuration: Math.round(average),\n    medianDuration: Math.round(median),\n    p95Duration: Math.round(p95),\n    slowestJobs,\n    slowestSteps,\n    cacheSavings: estimateCacheSavings(stepDurations),\n    recommendations,\n  };\n}\n\nfunction generateOptimizationRecommendations(\n  avgDuration: number,\n  slowestJobs: Array\u003C{ name: string; avgDuration: number }>,\n  slowestSteps: Array\u003C{ jobName: string; stepName: string; avgDuration: number }>\n): string[] {\n  const recommendations: string[] = [];\n\n  if (avgDuration > 600000) {\n    recommendations.push(\n      'Pipeline average exceeds 10 minutes. Consider increasing parallelization or enabling selective test execution.'\n    );\n  }\n\n  for (const job of slowestJobs) {\n    if (job.avgDuration > 300000) {\n      recommendations.push(\n        `Job \"${job.name}\" averages ${(job.avgDuration \u002F 60000).toFixed(1)}min. Consider splitting into shards.`\n      );\n    }\n  }\n\n  for (const step of slowestSteps) {\n    if (step.stepName.toLowerCase().includes('install') && step.avgDuration > 60000) {\n      recommendations.push(\n        `Step \"${step.stepName}\" in \"${step.jobName}\" takes ${(step.avgDuration \u002F 1000).toFixed(0)}s. Verify dependency caching is working correctly.`\n      );\n    }\n    if (step.stepName.toLowerCase().includes('build') && step.avgDuration > 120000) {\n      recommendations.push(\n        `Build step in \"${step.jobName}\" takes ${(step.avgDuration \u002F 1000).toFixed(0)}s. Consider incremental builds or build caching with Turborepo.`\n      );\n    }\n  }\n\n  return recommendations;\n}\n\nfunction estimateCacheSavings(stepDurations: Map\u003Cstring, number[]>): number {\n  let savings = 0;\n\n  for (const [key, times] of stepDurations) {\n    if (key.toLowerCase().includes('install') || key.toLowerCase().includes('cache')) {\n      if (times.length > 1) {\n        const first = times[0];\n        const subsequent = times.slice(1);\n        const avgSubsequent =\n          subsequent.reduce((a, b) => a + b, 0) \u002F subsequent.length;\n        if (first > avgSubsequent * 2) {\n          savings += (first - avgSubsequent) * subsequent.length;\n        }\n      }\n    }\n  }\n\n  return Math.round(savings);\n}\n```\n\n## Critical Path Optimization\n\n### Understanding the Pipeline Dependency Graph\n\nThe critical path is the longest sequential chain of jobs in your pipeline. Parallelizing jobs off the critical path does not reduce total duration. Focus optimization on the critical path:\n\n```typescript\n\u002F\u002F scripts\u002Fcritical-path.ts\n\ninterface JobNode {\n  name: string;\n  estimatedDuration: number;\n  dependencies: string[];\n  canParallelize: boolean;\n  shardable: boolean;\n}\n\nexport function findCriticalPath(jobs: JobNode[]): {\n  path: JobNode[];\n  duration: number;\n  parallelizableSavings: number;\n} {\n  const jobMap = new Map(jobs.map((j) => [j.name, j]));\n  const memo = new Map\u003Cstring, number>();\n\n  function longestPath(jobName: string): number {\n    if (memo.has(jobName)) return memo.get(jobName)!;\n\n    const job = jobMap.get(jobName);\n    if (!job) return 0;\n\n    let maxDepDuration = 0;\n    for (const dep of job.dependencies) {\n      maxDepDuration = Math.max(maxDepDuration, longestPath(dep));\n    }\n\n    const total = maxDepDuration + job.estimatedDuration;\n    memo.set(jobName, total);\n    return total;\n  }\n\n  \u002F\u002F Find the endpoint of the critical path\n  let maxDuration = 0;\n  let criticalEndJob = '';\n\n  for (const job of jobs) {\n    const duration = longestPath(job.name);\n    if (duration > maxDuration) {\n      maxDuration = duration;\n      criticalEndJob = job.name;\n    }\n  }\n\n  \u002F\u002F Reconstruct the critical path by walking backwards\n  const path: JobNode[] = [];\n  let current = criticalEndJob;\n\n  while (current) {\n    const job = jobMap.get(current);\n    if (!job) break;\n    path.unshift(job);\n\n    let nextJob = '';\n    let nextDuration = 0;\n    for (const dep of job.dependencies) {\n      const depDuration = memo.get(dep) || 0;\n      if (depDuration > nextDuration) {\n        nextDuration = depDuration;\n        nextJob = dep;\n      }\n    }\n    current = nextJob;\n  }\n\n  const totalSequentialDuration = jobs.reduce(\n    (sum, job) => sum + job.estimatedDuration,\n    0\n  );\n  const criticalPathDuration = path.reduce(\n    (sum, job) => sum + job.estimatedDuration,\n    0\n  );\n\n  return {\n    path,\n    duration: criticalPathDuration,\n    parallelizableSavings: totalSequentialDuration - criticalPathDuration,\n  };\n}\n```\n\n## Best Practices\n\n1. **Measure before optimizing.** Profile your current pipeline to identify actual bottlenecks. The slowest step may not be what you expect. Use pipeline analytics to find the real performance killers before implementing any optimizations.\n\n2. **Use concurrency controls to cancel redundant runs.** When a developer pushes multiple commits in quick succession, cancel the in-progress run for the previous commit. The GitHub Actions `concurrency` key with `cancel-in-progress: true` handles this automatically.\n\n3. **Cache aggressively but validate cache correctness.** Cache node_modules, build artifacts, Playwright browsers, and Docker layers. But always verify that stale caches do not cause false-positive test results. Include lock file hashes in cache keys to invalidate on dependency changes.\n\n4. **Split E2E tests across more shards than unit tests.** E2E tests are typically 10-50x slower than unit tests. A suite that benefits from 2 unit test shards may need 4-8 E2E test shards for balanced execution times.\n\n5. **Run lint and type-check before tests.** These checks are fast (typically under 60 seconds) and catch a large class of issues. Running them before expensive test suites provides faster initial feedback when they fail.\n\n6. **Use fail-fast: false for sharded jobs.** When tests are split across shards, a failure in shard 1 should not cancel shard 3. You want complete results from all shards to see the full scope of failures.\n\n7. **Collect and store test timing data persistently.** Duration-based test splitting requires historical data to be effective. Collect timings after every CI run and cache them for future runs. Use exponential moving averages to smooth outliers.\n\n8. **Implement selective test execution with a safety net.** When change detection cannot determine which tests to run (e.g., for config file changes), default to running all tests. A false positive (running unnecessary tests) is always safer than a false negative (missing a regression).\n\n9. **Pre-warm caches on a schedule.** Run a scheduled workflow that installs dependencies and populates caches before the workday begins. This ensures the first PR of the day gets a cache hit instead of a cold start.\n\n10. **Monitor pipeline duration trends over time.** Track P50 and P95 pipeline durations weekly. A gradual increase of 30 seconds per week adds up to 25 minutes over a year. Catch regressions early with automated alerting on duration spikes.\n\n11. **Use service containers for integration test dependencies.** Instead of installing PostgreSQL or Redis in your workflow steps, use GitHub Actions service containers. They start in parallel with job setup and are ready when your tests need them.\n\n12. **Keep Docker images minimal for CI runners.** If your CI uses custom Docker images, minimize their size. Every megabyte of image that needs to be pulled adds to every pipeline run. Use multi-stage builds and slim base images.\n\n## Anti-Patterns to Avoid\n\n**Running all tests on every PR regardless of changes.** A documentation-only PR does not need E2E tests. A frontend-only change does not need backend integration tests. Implement change detection to skip unnecessary work and save both time and compute resources.\n\n**Using sequential job execution when parallelism is possible.** If unit tests and E2E tests have no dependency on each other, run them in parallel. Restructure your pipeline so that only actual data dependencies create sequential chains.\n\n**Caching too broadly or too narrowly.** Caching `node_modules` without including the lock file hash in the cache key means stale dependencies persist. Caching only `node_modules` but not Playwright browsers means downloading 100+ MB of browser binaries on every run.\n\n**Splitting tests by file count instead of duration.** Ten test files may take 10 seconds or 10 minutes depending on their content. Always use duration-based splitting when timing data is available to ensure balanced shard execution times.\n\n**Ignoring the critical path.** Parallelizing jobs that are already off the critical path does not reduce total pipeline duration. Identify and focus optimization on the jobs that form the longest sequential chain.\n\n**Not canceling redundant pipeline runs.** Without concurrency controls, every push creates a new pipeline run. Three pushes in 5 minutes create three full runs, wasting resources and delaying results for other developers waiting in the queue.\n\n**Hard-coding shard counts.** A static 4-shard configuration may be wasteful for small PRs and insufficient for full test runs on main. Dynamically determine shard counts based on the scope of changes detected.\n\n**Skipping tests to speed up the pipeline.** Disabling tests is not optimization; it is risk accumulation. Optimize the tests themselves (reduce flakiness, parallelize, cache) rather than removing them from the pipeline.\n\n## Debugging Tips\n\n**Cache misses on every run.** Verify that the cache key includes only deterministic inputs. Environment variables, timestamps, and random values in cache keys cause every run to miss. Use `hashFiles()` on lock files and configuration files for consistent keys.\n\n**Sharded tests produce inconsistent results.** Test isolation issues become visible under sharding because tests run in different orders and different processes. Identify tests that depend on shared state (global variables, database records, file system artifacts) and fix them to be truly independent.\n\n**Selective test execution misses a regression.** Review your change-to-test mapping configuration. Common gaps include: shared utility files that are not mapped to all their consumers, configuration changes that should trigger full runs, and transitive dependencies that are not tracked in the change map.\n\n**Pipeline duration increases after adding caching.** This can happen when cache save and restore steps take longer than the operation they are caching. Very small caches (under 10 MB) may not be worth the overhead. Profile the cache steps themselves to verify they provide net savings.\n\n**Test timings file grows unbounded.** Prune entries for test files that no longer exist. When files are renamed or deleted, their timing entries remain in the file indefinitely. Add a cleanup step that removes entries for files not present in the current codebase.\n\n**GitHub Actions job matrix generates too many combinations.** Matrix strategies with multiple dimensions (OS x Node version x shard) can produce dozens of jobs. Use `include` and `exclude` to limit combinations to only those that provide unique value.\n\n**Jobs fail with \"No space left on device.\"** CI runners have limited disk space. Large caches, Docker images, and build artifacts can exhaust the available storage. Add cleanup steps between resource-intensive operations, use slimmer base images, and prune unnecessary artifacts before running tests.\n",{"name":5,"tags":24,"agents":32,"author":45,"domains":46,"license":15,"version":48,"languages":49,"frameworks":54,"description":7,"testingTypes":58},[25,26,27,28,29,30,31],"ci-cd","pipeline","optimization","parallelization","test-splitting","caching","github-actions",[33,34,35,36,37,38,39,40,41,42,43,44],"claude-code","cursor","github-copilot","windsurf","codex","aider","continue","cline","zed","bolt","gemini-cli","amp","Pramod",[47],"devops","1.0.0",[50,51,52,53],"typescript","javascript","python","java",[55,56,57],"jest","vitest","playwright",[59],"code-quality","main","eac438ea9216a7cb8a570cab803eb01eeafc6793","skills\u002Fqa\u002Fci-pipeline-optimizer","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m","https:\u002F\u002Fid-preview--d5bef06d-6c9e-4563-a3d4-a855b9d61610.lovable.app\u002F","Eu sou Antonio B. B. Ndombe \r\nEu sou um Pesquisador e desenvolvedor de sistema autonomia cognitivos, fundador de sistemas ODCs ( organismos digital conscientes)","2026-11-04 06:53:39",{"source":8,"skillName":5,"commands":69,"prompt":79,"zipAvailable":80},[70,73,76],{"tool":71,"command":72},"npx","npx skills add GrupoANDevelopment-m\u002Falberto-ai --skill \"CI Pipeline Optimizer\"",{"tool":74,"command":75},"bunx","bunx skills add GrupoANDevelopment-m\u002Falberto-ai --skill \"CI Pipeline Optimizer\"",{"tool":77,"command":78},"pnpm","pnpm dlx skills add GrupoANDevelopment-m\u002Falberto-ai --skill \"CI Pipeline Optimizer\"","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai\u002Fblob\u002Fmain\u002Fskills\u002Fqa\u002Fci-pipeline-optimizer\u002FSKILL.md",true,[],[83,91,98,105,112,119],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":87,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":19},"01M7DASC6FSCJP49JZBSK0NY1Z","nemoclaw-user-reference","Describes the NemoClaw integration layer and blueprint architecture and how they orchestrate compatible agent sandboxes. Use when looking up architecture, agent integration, plugin structure, or blueprint design. Trigger keywords - nemoclaw architecture, nemoclaw agent architecture, nemoclaw plugin blueprint structure, nemoclaw vs openshell, which cli, nemoclaw cli, openshell cli, sandbox commands, nemoclaw cli commands, nemoclaw command reference, nemoclaw network policy, sandbox egress control operator approval, nemoclaw troubleshooting, nemoclaw debug sandbox issues.","Apache-2.0","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai\u002Fblob\u002Fmain\u002Fupstream\u002Fnemoclaw\u002Fskills\u002Fnemoclaw-user-reference\u002FSKILL.md",[],[],{"skillNo":92,"name":93,"nameZh":6,"description":94,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":87,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":19},"01M7DAS5S79VK33XFGVAM2TFWF","nemoclaw-user-overview","Explains how OpenClaw, OpenShell, and NemoClaw form the ecosystem, NemoClaw's position in the stack, what NemoClaw adds beyond the community sandbox, and when to prefer NemoClaw versus integrating OpenShell and OpenClaw directly. Use when users ask about the relationship between OpenClaw, OpenShell, and NemoClaw, or when to use NemoClaw versus OpenShell. Trigger keywords - nemoclaw ecosystem, openclaw openshell, nemoclaw vs openshell, sandboxed openclaw, how nemoclaw works, nemoclaw sandbox lifecycle blueprint, nemoclaw overview, openclaw always-on assistants, nvidia openshell, nvidia nemotron, nemoclaw release notes, nemoclaw changelog.","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai\u002Fblob\u002Fmain\u002Fupstream\u002Fnemoclaw\u002Fskills\u002Fnemoclaw-user-overview\u002FSKILL.md",[],[],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":87,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":19},"01M7DAS0PQEMET5MNGZ1ABSZVS","nemoclaw-user-monitor-sandbox","Inspects sandbox health, traces agent behavior, and diagnoses problems. Use when monitoring a running sandbox, debugging agent issues, or checking sandbox logs. Trigger keywords - monitor nemoclaw sandbox, debug nemoclaw agent issues.","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai\u002Fblob\u002Fmain\u002Fupstream\u002Fnemoclaw\u002Fskills\u002Fnemoclaw-user-monitor-sandbox\u002FSKILL.md",[],[],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":87,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":19},"01M7DARQYGF7E9A69VH6S3CGN9","nemoclaw-user-manage-sandboxes","Explains operational tasks after the quickstart: listing sandboxes, status and health checks, logs, diagnostics, port forwards, multiple sandboxes, credential reset, rebuilds, network presets, upgrades, and uninstall. Trigger keywords - manage nemoclaw sandboxes, nemoclaw status, nemoclaw list, nemoclaw dashboard port, nemoclaw rebuild, nemoclaw upgrade sandboxes, nemoclaw uninstall, sandbox mutability, sandbox runtime configuration, sandbox rebuild, nemoclaw backup, nemoclaw restore, workspace backup, openshell sandbox download upload, nemoclaw messaging channels, nemoclaw telegram, nemoclaw discord, nemoclaw slack, nemoclaw wechat, nemoclaw whatsapp, openshell channel messaging, nemoclaw workspace files, soul.md, user.md, identity.md, agents.md, sandbox persistence.","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai\u002Fblob\u002Fmain\u002Fupstream\u002Fnemoclaw\u002Fskills\u002Fnemoclaw-user-manage-sandboxes\u002FSKILL.md",[],[],{"skillNo":113,"name":114,"nameZh":6,"description":115,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":87,"licenseFileUrl":6,"sourceHtmlUrl":116,"categories":117,"keywords":118,"publishedAt":19},"01M7DARH8BRTG51TRCAB7YWDJ4","nemoclaw-user-manage-policy","Adds, removes, or modifies allowed endpoints in the sandbox policy. Use when customizing network policy, changing egress rules, or configuring sandbox endpoint access. Trigger keywords - customize nemoclaw network policy, sandbox egress policy configuration, nemoclaw integration policy examples, post-install policy setup, openshell approval workflow, policy preset, nemoclaw approve network requests, sandbox egress approval tui.","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai\u002Fblob\u002Fmain\u002Fupstream\u002Fnemoclaw\u002Fskills\u002Fnemoclaw-user-manage-policy\u002FSKILL.md",[],[],{"skillNo":120,"name":121,"nameZh":6,"description":122,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":87,"licenseFileUrl":6,"sourceHtmlUrl":123,"categories":124,"keywords":125,"publishedAt":19},"01M7DARAT62M2VR37590VCZ5NS","nemoclaw-user-get-started","Installs NemoClaw, launches a sandbox, and runs the first agent prompt. Use when onboarding, installing, or launching a NemoClaw sandbox for the first time. Trigger keywords - nemoclaw quickstart, install nemoclaw openclaw sandbox, nemohermes quickstart, hermes agent nemoclaw, run hermes openshell sandbox, nemoclaw prerequisites, nemoclaw supported platforms, nemoclaw hardware software, nemoclaw windows wsl2 setup, nemoclaw install windows docker desktop.","https:\u002F\u002Fgithub.com\u002FGrupoANDevelopment-m\u002Falberto-ai\u002Fblob\u002Fmain\u002Fupstream\u002Fnemoclaw\u002Fskills\u002Fnemoclaw-user-get-started\u002FSKILL.md",[],[]]