[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6EES7CFHBX4D08W6NS4RGC5":3,"skill-same:01M6EES7CFHBX4D08W6NS4RGC5":52},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":25,"linkOnly":26,"markdownContent":27,"markdownBody":27,"frontMatter":28,"refName":29,"commitSha":30,"skillDirectory":31,"forksCount":32,"repositoryHtmlUrl":33,"repositoryHomepageUrl":34,"ownerHtmlUrl":35,"ownerBlogUrl":6,"ownerDescription":36,"updatedAt":37,"install":38,"resources":51},"01M6EES7CFHBX4D08W6NS4RGC5","data-validator",null,"ComeOnOliver\u002Fskillshub","ComeOnOliver","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F43215937?v=4","01M6E5JNFECJAAVFE0W7SMWWD2","Haoran Shu","01M6E5JNNDQS7AJZDYFQ8TPZCK",67,"MIT","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002FTerminalSkills\u002Fskills\u002Fdata-validator\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[23,24],"data","data validator","2026-11-10 10:12:30",false,"# Data Validator\n\n## Overview\n\nPerform comprehensive data quality checks on datasets — validate schemas, detect anomalies, find duplicates, and enforce data contracts. Essential for ETL pipelines where bad data silently corrupts downstream analytics and dashboards.\n\n## Instructions\n\n### 1. Profile the dataset first\n\nBefore validating, understand the data:\n- Row count and column count\n- Data types per column (string, integer, float, date, boolean)\n- Null rates per column\n- Unique value counts and cardinality\n- Min\u002Fmax\u002Fmean for numeric columns\n- Date ranges for temporal columns\n\nPresent as a data profile summary:\n```\nDataset Profile: orders_export.csv\nRows: 142,847 | Columns: 12\n\n| Column        | Type    | Nulls  | Unique  | Sample Values          |\n|---------------|---------|--------|---------|------------------------|\n| order_id      | string  | 0%     | 142,847 | ORD-20260217-001       |\n| customer_id   | integer | 0.3%   | 28,491  | 10042, 10043           |\n| amount         | float   | 0%     | 8,234   | 29.99, 149.00          |\n| created_at    | date    | 0%     | 89,112  | 2026-02-17T14:23:01Z   |\n| status        | string  | 0%     | 5       | completed, pending      |\n```\n\n### 2. Run validation checks\n\nApply these checks systematically:\n\n**Completeness** — Are required fields populated?\n- Check null\u002Fempty rates against thresholds (e.g., email must be \u003C1% null)\n- Flag columns with unexpected null spikes\n\n**Uniqueness** — Are IDs actually unique?\n- Check primary key columns for duplicates\n- Report duplicate counts and sample duplicate rows\n\n**Type consistency** — Do values match expected types?\n- Dates that don't parse, numbers stored as strings\n- Mixed types in the same column\n\n**Range validity** — Are values within expected bounds?\n- Negative amounts, future dates, ages over 150\n- Enum columns with unexpected values\n\n**Referential integrity** — Do foreign keys match?\n- customer_id values that don't exist in the customers table\n- Orphaned records\n\n**Freshness** — Is the data up to date?\n- Most recent record timestamp vs current time\n- Gaps in time series data\n\n### 3. Report findings\n\nStructure results as a validation report:\n```\n## Data Validation Report\nDataset: orders_export.csv | Checked: 2026-02-17 14:30 UTC\n\n### ❌ Failed Checks (3)\n1. **Duplicate order_id** — 23 duplicate IDs found (0.016%)\n   Sample: ORD-20260215-4421 appears 3 times\n2. **Null customer_email** — 2.1% null (threshold: 1%)\n   Spike on 2026-02-15 (bulk import batch)\n3. **Future dates in created_at** — 7 rows have dates in 2027\n\n### ⚠️ Warnings (2)\n1. **Amount outliers** — 4 orders exceed $10,000 (review manually)\n2. **Status enum drift** — New value \"on_hold\" not in expected set\n\n### ✅ Passed Checks (8)\n- Primary key uniqueness (order_id): PASS (99.98%)\n- Amount non-negative: PASS\n- Date range valid: PASS (excluding 7 future dates)\n...\n```\n\n### 4. Suggest fixes\n\nFor each failed check, recommend a concrete action:\n- SQL query to find and fix duplicates\n- Transform to clean null values\n- Filter to remove invalid rows\n- Alert to set up for ongoing monitoring\n\n## Examples\n\n### Example 1: CSV Import Validation\n\n**Input:** \"Validate this customer export before importing to our new CRM\"\n\n**Output:**\n```\n## Validation Report: customers_export.csv\nRows: 8,234 | Columns: 9\n\n### ❌ Failed (2)\n1. **Email format invalid** — 142 rows (1.7%) have malformed emails\n   Pattern: missing @ symbol, spaces in domain\n   Fix: SELECT * FROM data WHERE email NOT LIKE '%_@_%.__%'\n\n2. **Phone duplicates** — 34 phone numbers shared by multiple customers\n   Likely data entry errors or shared business lines\n   Fix: Review and merge duplicate customer records\n\n### ✅ Passed (6)\n- customer_id unique: PASS\n- Required fields (name, email): PASS (99.8%)\n- Country codes valid ISO-3166: PASS\n- Created dates in range: PASS\n- No PII in notes field: PASS\n- UTF-8 encoding clean: PASS\n\nRecommendation: Fix the 142 invalid emails before import.\nThe phone duplicates are non-blocking but should be reviewed.\n```\n\n### Example 2: Pipeline Data Contract Check\n\n**Input:** \"Check if the analytics events from our tracking pipeline match the expected schema\"\n\n**Output:**\n```\n## Schema Contract Validation: events_2026_02_17.json\nEvents: 1.2M | Schema version: v3.2\n\n### ❌ Contract Violations (1)\n1. **Missing required field: session_id**\n   - 12,847 events (1.07%) missing session_id\n   - All from source: mobile-ios-sdk v4.1.0\n   - Started: 2026-02-16 09:00 UTC (correlates with SDK release)\n\n### ⚠️ Warnings (1)\n1. **New field detected: device_model**\n   - Not in schema v3.2 — likely added in SDK update\n   - 100% populated on mobile events, absent on web\n\n### ✅ Contract Compliance: 98.93%\n\nAction: Pin mobile-ios-sdk to v4.0.x or update schema to v3.3\nwith session_id as optional for mobile sources.\n```\n\n## Guidelines\n\n- Always profile before validating — you need baselines to detect anomalies\n- Distinguish between hard failures (blocks the pipeline) and warnings (review later)\n- Include sample rows for every finding — abstract stats are hard to act on\n- Suggest SQL or code fixes, not just descriptions of problems\n- For time series data, check for gaps and seasonality, not just latest timestamp\n- Track validation results over time to detect data drift\n- Be specific about which rows\u002Frecords are affected — \"23 duplicates\" beats \"some duplicates\"\n",{},"main","def8531e65114c0fca8fb8551c1871ee0eed705c","skills\u002FTerminalSkills\u002Fskills\u002Fdata-validator",24,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub","https:\u002F\u002Fskillshub.wtf","https:\u002F\u002Fgithub.com\u002FComeOnOliver","AI & Blockchain Engineer\r\n","2026-06-24 22:07:19",{"source":7,"skillName":5,"commands":39,"prompt":49,"zipAvailable":50},[40,43,46],{"tool":41,"command":42},"npx","npx skills add ComeOnOliver\u002Fskillshub --skill data-validator",{"tool":44,"command":45},"bunx","bunx skills add ComeOnOliver\u002Fskillshub --skill data-validator",{"tool":47,"command":48},"pnpm","pnpm dlx skills add ComeOnOliver\u002Fskillshub --skill data-validator","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002FTerminalSkills\u002Fskills\u002Fdata-validator\u002FSKILL.md",true,[],[53,67,77,86,92,99],{"skillNo":54,"name":55,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":56,"categories":57,"keywords":63,"publishedAt":25},"01M6FKJN13GMG4KY57J3QD8GY5","model-hierarchy-skill","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzscole\u002Fmodel-hierarchy-skill\u002Fmodel-hierarchy-skill\u002FSKILL.md",[58],{"categoryNo":59,"categoryName":60,"taxonomyType":61,"taxonomyTypeDesc":62},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[64,65,66],"model","model hierarchy","model hierarchy skill",{"skillNo":68,"name":69,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":70,"categories":71,"keywords":73,"publishedAt":25},"01M6FKJJJ6GCAMR4PPPRD12M5X","zoom-mcp","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fzoom-mcp\u002FSKILL.md",[72],{"categoryNo":59,"categoryName":60,"taxonomyType":61,"taxonomyTypeDesc":62},[74,75,76],"MCP","zoom","zoom mcp",{"skillNo":78,"name":79,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":82,"publishedAt":25},"01M6FKJE3KYT3KPEZMEEHYKA6K","zoom-apps-sdk","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fzoom-apps-sdk\u002FSKILL.md",[],[83,84,85,75],"sdk","zoom apps sdk","apps",{"skillNo":87,"name":88,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":25},"01M6FKJ9EDH9Q5VWDF8HAJRSP0","windows","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwindows\u002FSKILL.md",[],[88],{"skillNo":93,"name":94,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":98,"publishedAt":25},"01M6FKJ666BEEFC3AE9GJ72Q93","whiteboard","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwhiteboard\u002FSKILL.md",[97],{"categoryNo":59,"categoryName":60,"taxonomyType":61,"taxonomyTypeDesc":62},[94],{"skillNo":100,"name":101,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":25},"01M6FKJ21WFCDGTJQ2Q91T9DVW","websockets","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwebsockets\u002FSKILL.md",[],[101]]