[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M74WNW7X5TDFDH286B06EM72":3,"skill-same:01M74WNW7X5TDFDH286B06EM72":55},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":29,"commitSha":30,"skillDirectory":5,"forksCount":14,"repositoryHtmlUrl":31,"repositoryHomepageUrl":6,"ownerHtmlUrl":32,"ownerBlogUrl":33,"ownerDescription":6,"updatedAt":34,"install":35,"resources":48},"01M74WNW7X5TDFDH286B06EM72","databricks-core-workflow-a",null,"Execute Databricks primary workflow: Delta Lake ETL pipelines.\nUse when building data ingestion pipelines, implementing medallion architecture,\nor creating Delta Lake transformations.\nTrigger with phrases like \"databricks ETL\", \"delta lake pipeline\",\n\"medallion architecture\", \"databricks data pipeline\", \"bronze silver gold\".","SamuelChien\u002Fmega-skills-collection","SamuelChien","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F4652114?v=4","01M74KK34WVHA0P5ZYSJRDDFMG","Samuel Chien","01M74KK35FQN1N9VZ9EPTH3P2S",0,"MIT","https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fdatabricks-core-workflow-a\u002FSKILL.md",[],[],"2026-11-12 13:55:43",false,"---\nname: databricks-core-workflow-a\ndescription: |\n  Execute Databricks primary workflow: Delta Lake ETL pipelines.\n  Use when building data ingestion pipelines, implementing medallion architecture,\n  or creating Delta Lake transformations.\n  Trigger with phrases like \"databricks ETL\", \"delta lake pipeline\",\n  \"medallion architecture\", \"databricks data pipeline\", \"bronze silver gold\".\nallowed-tools: Read, Write, Edit, Bash(databricks:*), Grep\nversion: 1.0.0\nlicense: MIT\nauthor: Jeremy Longshore \u003Cjeremy@intentsolutions.io>\ncompatible-with: claude-code, codex, openclaw\n---\n# Databricks Core Workflow A: Delta Lake ETL\n\n## Contents\n- [Overview](#overview)\n- [Prerequisites](#prerequisites)\n- [Instructions](#instructions)\n- [Output](#output)\n- [Error Handling](#error-handling)\n- [Examples](#examples)\n- [Resources](#resources)\n\n## Overview\nBuild production Delta Lake ETL pipelines using medallion architecture (Bronze -> Silver -> Gold).\n\n## Prerequisites\n- Completed `databricks-install-auth` setup\n- Understanding of Delta Lake concepts\n- Unity Catalog configured (recommended)\n\n## Medallion Architecture\n```\nRaw Sources -> Bronze (Raw\u002FLanding) -> Silver (Cleaned\u002FBusiness Logic) -> Gold (Aggregated\u002FAnalytics Ready)\n```\n\n## Instructions\n\n### Step 1: Bronze Layer - Raw Ingestion\nIngest raw data with metadata columns (`_ingested_at`, `_source_file`). Use `mergeSchema` for schema evolution. Use Auto Loader (`cloudFiles`) for streaming ingestion with schema inference.\n\n### Step 2: Silver Layer - Data Cleansing\nRead from Bronze using Change Data Feed. Apply transformations: trim\u002Flowercase strings, parse timestamps, hash PII, filter nulls, generate surrogate keys. Merge into Silver with upsert pattern.\n\n### Step 3: Gold Layer - Business Aggregations\nAggregate Silver data by business dimensions and time grain. Use partition-level overwrites for efficient updates.\n\n### Step 4: Delta Live Tables (Optional)\nDeclarative pipeline with `@dlt.table` decorators and data quality expectations (`@dlt.expect_or_drop`).\n\nSee [detailed implementation](${CLAUDE_SKILL_DIR}\u002Freferences\u002Fimplementation.md) for complete Bronze\u002FSilver\u002FGold pipeline code, Auto Loader config, DLT pipeline, and orchestration example.\n\n## Output\n- Bronze layer with raw data and ingestion metadata\n- Silver layer with cleansed, deduplicated data\n- Gold layer with business aggregations\n- Delta Lake tables with ACID transactions\n\n## Error Handling\n| Error | Cause | Solution |\n|-------|-------|----------|\n| Schema mismatch | Source schema changed | Use `mergeSchema` option |\n| Duplicate records | Missing deduplication | Add merge logic with primary keys |\n| Null values | Data quality issues | Add expectations\u002Ffilters in Silver |\n| Memory errors | Large aggregations | Increase cluster size or partition data |\n\n## Examples\n\n### Quick Pipeline Run\n```python\n# Full medallion pipeline\nbronze.ingest_to_bronze(spark, \"\u002Fmnt\u002Flanding\u002Forders\u002F\", \"catalog.bronze.orders\")\nsilver.transform_to_silver(spark, \"catalog.bronze.orders\", \"catalog.silver.orders\", primary_keys=[\"order_id\"])\ngold.aggregate_to_gold(spark, \"catalog.silver.orders\", \"catalog.gold.metrics\", group_by_columns=[\"region\"])\n```\n\n## Resources\n- [Delta Lake Guide](https:\u002F\u002Fdocs.databricks.com\u002Fdelta\u002Findex.html)\n- [Medallion Architecture](https:\u002F\u002Fwww.databricks.com\u002Fglossary\u002Fmedallion-architecture)\n- [Delta Live Tables](https:\u002F\u002Fdocs.databricks.com\u002Fdelta-live-tables\u002Findex.html)\n- [Auto Loader](https:\u002F\u002Fdocs.databricks.com\u002Fingestion\u002Fauto-loader\u002Findex.html)\n\n## Next Steps\nFor ML workflows, see `databricks-core-workflow-b`.","# Databricks Core Workflow A: Delta Lake ETL\n\n## Contents\n- [Overview](#overview)\n- [Prerequisites](#prerequisites)\n- [Instructions](#instructions)\n- [Output](#output)\n- [Error Handling](#error-handling)\n- [Examples](#examples)\n- [Resources](#resources)\n\n## Overview\nBuild production Delta Lake ETL pipelines using medallion architecture (Bronze -> Silver -> Gold).\n\n## Prerequisites\n- Completed `databricks-install-auth` setup\n- Understanding of Delta Lake concepts\n- Unity Catalog configured (recommended)\n\n## Medallion Architecture\n```\nRaw Sources -> Bronze (Raw\u002FLanding) -> Silver (Cleaned\u002FBusiness Logic) -> Gold (Aggregated\u002FAnalytics Ready)\n```\n\n## Instructions\n\n### Step 1: Bronze Layer - Raw Ingestion\nIngest raw data with metadata columns (`_ingested_at`, `_source_file`). Use `mergeSchema` for schema evolution. Use Auto Loader (`cloudFiles`) for streaming ingestion with schema inference.\n\n### Step 2: Silver Layer - Data Cleansing\nRead from Bronze using Change Data Feed. Apply transformations: trim\u002Flowercase strings, parse timestamps, hash PII, filter nulls, generate surrogate keys. Merge into Silver with upsert pattern.\n\n### Step 3: Gold Layer - Business Aggregations\nAggregate Silver data by business dimensions and time grain. Use partition-level overwrites for efficient updates.\n\n### Step 4: Delta Live Tables (Optional)\nDeclarative pipeline with `@dlt.table` decorators and data quality expectations (`@dlt.expect_or_drop`).\n\nSee [detailed implementation](${CLAUDE_SKILL_DIR}\u002Freferences\u002Fimplementation.md) for complete Bronze\u002FSilver\u002FGold pipeline code, Auto Loader config, DLT pipeline, and orchestration example.\n\n## Output\n- Bronze layer with raw data and ingestion metadata\n- Silver layer with cleansed, deduplicated data\n- Gold layer with business aggregations\n- Delta Lake tables with ACID transactions\n\n## Error Handling\n| Error | Cause | Solution |\n|-------|-------|----------|\n| Schema mismatch | Source schema changed | Use `mergeSchema` option |\n| Duplicate records | Missing deduplication | Add merge logic with primary keys |\n| Null values | Data quality issues | Add expectations\u002Ffilters in Silver |\n| Memory errors | Large aggregations | Increase cluster size or partition data |\n\n## Examples\n\n### Quick Pipeline Run\n```python\n# Full medallion pipeline\nbronze.ingest_to_bronze(spark, \"\u002Fmnt\u002Flanding\u002Forders\u002F\", \"catalog.bronze.orders\")\nsilver.transform_to_silver(spark, \"catalog.bronze.orders\", \"catalog.silver.orders\", primary_keys=[\"order_id\"])\ngold.aggregate_to_gold(spark, \"catalog.silver.orders\", \"catalog.gold.metrics\", group_by_columns=[\"region\"])\n```\n\n## Resources\n- [Delta Lake Guide](https:\u002F\u002Fdocs.databricks.com\u002Fdelta\u002Findex.html)\n- [Medallion Architecture](https:\u002F\u002Fwww.databricks.com\u002Fglossary\u002Fmedallion-architecture)\n- [Delta Live Tables](https:\u002F\u002Fdocs.databricks.com\u002Fdelta-live-tables\u002Findex.html)\n- [Auto Loader](https:\u002F\u002Fdocs.databricks.com\u002Fingestion\u002Fauto-loader\u002Findex.html)\n\n## Next Steps\nFor ML workflows, see `databricks-core-workflow-b`.",{"name":5,"author":24,"license":15,"version":25,"description":26,"allowed-tools":27,"compatible-with":28},"Jeremy Longshore \u003Cjeremy@intentsolutions.io>","1.0.0","Execute Databricks primary workflow: Delta Lake ETL pipelines.\nUse when building data ingestion pipelines, implementing medallion architecture,\nor creating Delta Lake transformations.\nTrigger with phrases like \"databricks ETL\", \"delta lake pipeline\",\n\"medallion architecture\", \"databricks data pipeline\", \"bronze silver gold\".\n","Read, Write, Edit, Bash(databricks:*), Grep","claude-code, codex, openclaw","main","c2b1fdccc779e99ac814e55fb69ca8d71215dd0f","https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection","https:\u002F\u002Fgithub.com\u002FSamuelChien","http:\u002F\u002Fsamuelchien.com\u002F","2026-05-26 08:27:14",{"source":8,"skillName":5,"commands":36,"prompt":46,"zipAvailable":47},[37,40,43],{"tool":38,"command":39},"npx","npx skills add SamuelChien\u002Fmega-skills-collection --skill databricks-core-workflow-a",{"tool":41,"command":42},"bunx","bunx skills add SamuelChien\u002Fmega-skills-collection --skill databricks-core-workflow-a",{"tool":44,"command":45},"pnpm","pnpm dlx skills add SamuelChien\u002Fmega-skills-collection --skill databricks-core-workflow-a","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fdatabricks-core-workflow-a\u002FSKILL.md",true,[49],{"path":50,"resourceType":51,"resourceTypeDesc":52,"sizeBytes":53,"sourceHtmlUrl":54},"references\u002Fimplementation.md","REFERENCE","参考文档",5170,"https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fdatabricks-core-workflow-a\u002Freferences\u002Fimplementation.md",[56,63,70,77,84,92],{"skillNo":57,"name":58,"nameZh":6,"description":59,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":19},"01M76R4WZQZAXGGXMQCA0FZ716","zustand-store-ts","Create Zustand stores with TypeScript, subscribeWithSelector middleware, and proper state\u002Faction separation. Use when building React state management, creating global stores, or implementing reactive state patterns with Zustand.","https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fzustand-store-ts\u002FSKILL.md",[],[],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":67,"categories":68,"keywords":69,"publishedAt":19},"01M76R4T9KENDH7FENGZQHBW4G","zustand-store-creator","Zustand Store Creator - Auto-activating skill for Frontend Development.\nTriggers on: zustand store creator, zustand store creator\nPart of the Frontend Development skill category.","https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fzustand-store-creator\u002FSKILL.md",[],[],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":19},"01M76R3YWN84B7SANJ52X799ZE","zero-trust-config-helper","Zero Trust Config Helper - Auto-activating skill for Security Advanced.\nTriggers on: zero trust config helper, zero trust config helper\nPart of the Security Advanced skill category.","https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fzero-trust-config-helper\u002FSKILL.md",[],[],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":19},"01M76R3HNSV2AEYSKHM915GJ58","zapier-integration-helper","Zapier Integration Helper - Auto-activating skill for Business Automation.\nTriggers on: zapier integration helper, zapier integration helper\nPart of the Business Automation skill category.","https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fzapier-integration-helper\u002FSKILL.md",[],[],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":88,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":19},"01M76R3DJSKAT4FA7DV4BF3GCE","zai-cli","Execute z.AI CLI providing vision, search, reader, and GitHub exploration via CLI and MCP.\nUse when user needs image\u002Fvideo analysis, OCR, UI-to-code conversion, error diagnosis,\nreal-time web search, web page to markdown extraction, or GitHub code exploration.\nTrigger with phrases like \"analyze this image\", \"search the web for\", \"read this page\",\n\"explore this repo\", or \"use zai\". Requires Z_AI_API_KEY.","Apache-2.0","https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fzai-cli\u002FSKILL.md",[],[],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":19},"01M76R2KAM699XE9AJVAWE4H2E","yt-research","Research competitor YouTube channels, niches, and trending topics for your content strategy.\nUse this skill whenever the user says \"research channels\", \"analyze competitors\", \"find trending topics\",\n\"niche analysis\", \"competitive research\", \"what are other creators doing\", \"scrape YouTube channels\",\nor wants to understand the competitive landscape for a specific tool or topic area. Use when working with yt research. Trigger with 'yt', 'research'.","https:\u002F\u002Fgithub.com\u002FSamuelChien\u002Fmega-skills-collection\u002Fblob\u002Fmain\u002Fyt-research\u002FSKILL.md",[],[]]