[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6CWJS6EPMRDS7VZ4JG1DRDW":3,"skill-same:01M6CWJS6EPMRDS7VZ4JG1DRDW":79},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":55,"commitSha":56,"skillDirectory":57,"forksCount":58,"repositoryHtmlUrl":59,"repositoryHomepageUrl":60,"ownerHtmlUrl":61,"ownerBlogUrl":62,"ownerDescription":63,"updatedAt":64,"install":65,"resources":78},"01M6CWJS6EPMRDS7VZ4JG1DRDW","dataset-onboard",null,"Onboard a new raw dataset into the data platform — sniff the schema, generate a profiling notebook, create the ingestion job, and add an entry to the data dictionary. Use when adding a new data source, when a partner delivers a new CSV\u002FParquet drop, or when asked to \"onboard the {name} dataset\".","zakelfassi\u002Fskills-driven-development","zakelfassi","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F1274085?v=4","01M6CWHB4BNG8NYGSW0YS08B7E","Zak El Fassi","01M6CWHB8QCK700M6B2S6Y1QPD",18,"MIT","https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development\u002Fblob\u002Fmain\u002Fexamples\u002Fdata-pipeline\u002Fskills\u002Fdataset-onboard\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"dataset onboard","onboard","use","orders","time","drop","structure","notebook","datasets","ingestion","schema","create","session","returns","profiling","inventory","data","csv","parquet","generate","2026-11-10 10:12:30",false,"---\nname: dataset-onboard\ndescription: Onboard a new raw dataset into the data platform — sniff the schema, generate a profiling notebook, create the ingestion job, and add an entry to the data dictionary. Use when adding a new data source, when a partner delivers a new CSV\u002FParquet drop, or when asked to \"onboard the {name} dataset\".\nmetadata:\n  forged-by: claude-agent\n  forged-from: session-2026-02-12\n  forged-reason: \"Onboarded orders, returns, and inventory datasets in one week; schema sniff + profiling + ingestion job had identical structure each time\"\n  usage-count: \"11\"\n  last-used: \"2026-05-20\"\n---\n\n# Dataset Onboard\n\nBring a new raw dataset into the platform with schema documentation, profiling, and an ingestion job.\n\n## Inputs\n- Dataset name (snake_case, e.g., `customer_events`)\n- Source format (`csv`, `parquet`, `json`, `avro`)\n- Source path or URI (S3, GCS, local mount, API endpoint)\n- Expected frequency (`daily`, `weekly`, `on-demand`)\n- Owner team and contact\n\n## Steps\n\n1. **Schema sniff**\n   ```python\n   import pandas as pd\n   df = pd.read_csv(\"{source_path}\", nrows=1000)   # or read_parquet, etc.\n   print(df.dtypes)\n   print(df.describe(include=\"all\"))\n   print(df.isnull().sum() \u002F len(df))              # null rates\n   ```\n   Document:\n   - Column names, inferred types, null rates, example values\n   - Detected anomalies (mixed types, encoding issues, unexpected nulls)\n\n2. **Create the data dictionary entry**\n   Edit `docs\u002Fdata-dictionary\u002F{dataset_name}.md`:\n   ```markdown\n   # {DatasetName}\n\n   **Owner:** {team}  **Contact:** {email}\n   **Source:** {uri}  **Frequency:** {frequency}\n\n   | Column | Type | Nullable | Description |\n   |--------|------|----------|-------------|\n   | ...    | ...  | ...      | ...         |\n   ```\n\n3. **Generate the profiling notebook**\n   ```bash\n   cp templates\u002Fprofiling-notebook.ipynb \\\n      notebooks\u002Fprofiling\u002F{dataset_name}_profile.ipynb\n   ```\n   Edit the notebook to use the correct source path and column list.\n   Run it to confirm it completes without errors.\n\n4. **Create the ingestion job**\n   ```\n   pipelines\u002Fingestion\u002F{dataset_name}\u002F\n   ├── ingest.py          # main ingestion script\n   ├── schema.py          # column definitions and type coercion\n   ├── config.yaml        # source path, schedule, destination table\n   └── tests\u002F\n       └── test_ingest.py # unit test with a small fixture file\n   ```\n   The ingestion script must be **idempotent** (re-running on the same input produces the same output; no duplicate rows).\n\n5. **Register in the scheduler**\n   Add a DAG entry in `dags\u002F{dataset_name}_ingest.py` (Airflow) or a flow in `flows\u002F` (Prefect).\n   Set the schedule to match `{frequency}`.\n\n6. **Test the ingestion**\n   ```bash\n   python -m pytest pipelines\u002Fingestion\u002F{dataset_name}\u002Ftests\u002F -v\n   python pipelines\u002Fingestion\u002F{dataset_name}\u002Fingest.py --dry-run\n   ```\n\n7. **Run a data quality gate** (invoke `data-quality-gate` skill)\n   Add at minimum: null checks for required columns, row-count sanity check.\n\n## Conventions\n- Raw datasets land in the `raw\u002F` schema\u002Flayer; never write to `staging\u002F` or `marts\u002F` from an ingestion job\n- All ingestion jobs accept `--dry-run` and `--date` flags\n- Dataset names are snake_case; tables follow the same naming\n- Profiling notebooks live in `notebooks\u002Fprofiling\u002F`; they are committed\n\n## Edge Cases\n- **Malformed source file:** Log the error with row number, skip the bad row, emit a `data_quality_alert` metric. Never silently swallow rows.\n- **Schema drift (columns added\u002Fremoved):** The ingestion job must compare the inbound schema to `schema.py` and fail fast on unexpected changes, rather than silently loading partial data.\n- **Large datasets (>1GB):** Use chunked reads (`chunksize` in pandas, or native Parquet partitioning); test with a 10k-row sample first.\n- **API source with rate limits:** Add exponential back-off and a `--resume-from` flag that uses a checkpoint file.\n","# Dataset Onboard\n\nBring a new raw dataset into the platform with schema documentation, profiling, and an ingestion job.\n\n## Inputs\n- Dataset name (snake_case, e.g., `customer_events`)\n- Source format (`csv`, `parquet`, `json`, `avro`)\n- Source path or URI (S3, GCS, local mount, API endpoint)\n- Expected frequency (`daily`, `weekly`, `on-demand`)\n- Owner team and contact\n\n## Steps\n\n1. **Schema sniff**\n   ```python\n   import pandas as pd\n   df = pd.read_csv(\"{source_path}\", nrows=1000)   # or read_parquet, etc.\n   print(df.dtypes)\n   print(df.describe(include=\"all\"))\n   print(df.isnull().sum() \u002F len(df))              # null rates\n   ```\n   Document:\n   - Column names, inferred types, null rates, example values\n   - Detected anomalies (mixed types, encoding issues, unexpected nulls)\n\n2. **Create the data dictionary entry**\n   Edit `docs\u002Fdata-dictionary\u002F{dataset_name}.md`:\n   ```markdown\n   # {DatasetName}\n\n   **Owner:** {team}  **Contact:** {email}\n   **Source:** {uri}  **Frequency:** {frequency}\n\n   | Column | Type | Nullable | Description |\n   |--------|------|----------|-------------|\n   | ...    | ...  | ...      | ...         |\n   ```\n\n3. **Generate the profiling notebook**\n   ```bash\n   cp templates\u002Fprofiling-notebook.ipynb \\\n      notebooks\u002Fprofiling\u002F{dataset_name}_profile.ipynb\n   ```\n   Edit the notebook to use the correct source path and column list.\n   Run it to confirm it completes without errors.\n\n4. **Create the ingestion job**\n   ```\n   pipelines\u002Fingestion\u002F{dataset_name}\u002F\n   ├── ingest.py          # main ingestion script\n   ├── schema.py          # column definitions and type coercion\n   ├── config.yaml        # source path, schedule, destination table\n   └── tests\u002F\n       └── test_ingest.py # unit test with a small fixture file\n   ```\n   The ingestion script must be **idempotent** (re-running on the same input produces the same output; no duplicate rows).\n\n5. **Register in the scheduler**\n   Add a DAG entry in `dags\u002F{dataset_name}_ingest.py` (Airflow) or a flow in `flows\u002F` (Prefect).\n   Set the schedule to match `{frequency}`.\n\n6. **Test the ingestion**\n   ```bash\n   python -m pytest pipelines\u002Fingestion\u002F{dataset_name}\u002Ftests\u002F -v\n   python pipelines\u002Fingestion\u002F{dataset_name}\u002Fingest.py --dry-run\n   ```\n\n7. **Run a data quality gate** (invoke `data-quality-gate` skill)\n   Add at minimum: null checks for required columns, row-count sanity check.\n\n## Conventions\n- Raw datasets land in the `raw\u002F` schema\u002Flayer; never write to `staging\u002F` or `marts\u002F` from an ingestion job\n- All ingestion jobs accept `--dry-run` and `--date` flags\n- Dataset names are snake_case; tables follow the same naming\n- Profiling notebooks live in `notebooks\u002Fprofiling\u002F`; they are committed\n\n## Edge Cases\n- **Malformed source file:** Log the error with row number, skip the bad row, emit a `data_quality_alert` metric. Never silently swallow rows.\n- **Schema drift (columns added\u002Fremoved):** The ingestion job must compare the inbound schema to `schema.py` and fail fast on unexpected changes, rather than silently loading partial data.\n- **Large datasets (>1GB):** Use chunked reads (`chunksize` in pandas, or native Parquet partitioning); test with a 10k-row sample first.\n- **API source with rate limits:** Add exponential back-off and a `--resume-from` flag that uses a checkpoint file.\n",{"name":5,"metadata":49,"description":7},{"forged-by":50,"last-used":51,"forged-from":52,"usage-count":53,"forged-reason":54},"claude-agent","2026-05-20","session-2026-02-12","11","Onboarded orders, returns, and inventory datasets in one week; schema sniff + profiling + ingestion job had identical structure each time","main","e959db0147cafe70b738efbd5a128985763fd0df","examples\u002Fdata-pipeline\u002Fskills\u002Fdataset-onboard",1,"https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development","https:\u002F\u002Fskdd.dev","https:\u002F\u002Fgithub.com\u002Fzakelfassi","https:\u002F\u002Fzakelfassi.com","I build systems and write about what they do to us. Founder @TalkAndComment.","2026-11-06 14:19:15",{"source":8,"skillName":5,"commands":66,"prompt":76,"zipAvailable":77},[67,70,73],{"tool":68,"command":69},"npx","npx skills add zakelfassi\u002Fskills-driven-development --skill dataset-onboard",{"tool":71,"command":72},"bunx","bunx skills add zakelfassi\u002Fskills-driven-development --skill dataset-onboard",{"tool":74,"command":75},"pnpm","pnpm dlx skills add zakelfassi\u002Fskills-driven-development --skill dataset-onboard","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development\u002Fblob\u002Fmain\u002Fexamples\u002Fdata-pipeline\u002Fskills\u002Fdataset-onboard\u002FSKILL.md",true,[],[80,97,103,126,142,164],{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":87,"publishedAt":44},"01M6CWKMXZDBQ8TK30ZV1X2KM7","skillforge","Create or update a reusable agent skill. Use when you notice a repeated pattern, when a workflow should be persisted for future sessions, or when asked to forge\u002Fcreate\u002Fscaffold a new skill.","https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development\u002Fblob\u002Fmain\u002Fskillforge\u002FSKILL.md",[86],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[82,88,89,90,91,92,26,93,94,35,95,96],"workflow","agentskills","pattern","new skill","reusable","forge","update","scaffold","sessions",{"skillNo":98,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":102,"publishedAt":44},"01M6CWKJF5Y1PQBZ4QWA9DDES1","https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development\u002Fblob\u002Fmain\u002Fplugins\u002Fskdd-claude\u002Fskills\u002Fskillforge\u002FSKILL.md",[101],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[82,88,93,96,89,90,91,92,26,94,35,95],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":113,"publishedAt":44},"01M6CWKFTNP9RBAD9SBEKRVMR2","deploy-preview","Deploy a preview branch to the staging environment for review. Use when a feature branch is ready for visual review, when someone asks to \"deploy this for review\", or before opening a PR.","https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development\u002Fblob\u002Fmain\u002Fexamples\u002Fwebapp-starter\u002Fskills\u002Fdeploy-preview\u002FSKILL.md",[109,112],{"categoryNo":110,"categoryName":111,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000011","运维与部署",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[114,115,116,117,118,28,119,120,26,36,121,122,123,124,125],"deploy preview","deploy","preview","visual","branches","ready","opening a pr","environment","branch","pr","codex","review",{"skillNo":127,"name":128,"nameZh":6,"description":129,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":130,"categories":131,"keywords":133,"publishedAt":44},"01M6CWKCSCQ6PV8TVKV2MDRAHH","component-scaffold","Create a new React component with co-located test, story, and styles. Use when creating new UI components, when asked to scaffold\u002Fgenerate a component, or when building new pages.","https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development\u002Fblob\u002Fmain\u002Fexamples\u002Fwebapp-starter\u002Fskills\u002Fcomponent-scaffold\u002FSKILL.md",[132],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[134,135,95,26,136,30,137,36,138,139,43,35,140,28,141],"component scaffold","component","components","Story","ui","file","react","test",{"skillNo":143,"name":144,"nameZh":6,"description":145,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":146,"categories":147,"keywords":149,"publishedAt":44},"01M6CWKA7S16TW2KZZ7ADK9YSA","bug-triage","Triage a bug report into an actionable GitHub issue with severity, reproduction steps, and assignment. Use when a bug is reported, when error logs need investigation, or when asked to file\u002Ftriage an issue.","https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development\u002Fblob\u002Fmain\u002Fexamples\u002Fwebapp-starter\u002Fskills\u002Fbug-triage\u002FSKILL.md",[148],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[150,151,152,153,139,154,155,156,157,158,159,160,161,26,162,36,163],"triage","bug","bug triage","logs","report","toolkit","bug report","github","severity","project","issue","github issue","ops","investigation",{"skillNo":165,"name":166,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":179,"publishedAt":44},"01M6CWK6Y4VB6HFNKP84WA3C6G","api-endpoint","Scaffold a REST API endpoint with route, controller, validation, and tests. Use when creating new API endpoints, adding resources, or when asked to build a new backend route.","https:\u002F\u002Fgithub.com\u002Fzakelfassi\u002Fskills-driven-development\u002Fblob\u002Fmain\u002Fexamples\u002Fwebapp-starter\u002Fskills\u002Fapi-endpoint\u002FSKILL.md",[170,173,174],{"categoryNo":171,"categoryName":172,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":175,"categoryName":176,"taxonomyType":177,"taxonomyTypeDesc":178},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[180,181,182,183,90,28,30,184,185,186,26,36,187,188,189,95,124],"endpoint","api","api endpoint","route","build","endpoints","backend","controller","REST","validation"]