[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6CSDYH24NS9TT5R54HTEW6J":3,"skill-same:01M6CSDYH24NS9TT5R54HTEW6J":168},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":28,"publishedAt":45,"linkOnly":46,"markdownContent":47,"markdownBody":48,"frontMatter":49,"refName":52,"commitSha":53,"skillDirectory":54,"forksCount":55,"repositoryHtmlUrl":56,"repositoryHomepageUrl":56,"ownerHtmlUrl":57,"ownerBlogUrl":6,"ownerDescription":58,"updatedAt":59,"install":60,"resources":73},"01M6CSDYH24NS9TT5R54HTEW6J","cc-skill-clickhouse-io",null,"ClickHouse database patterns, query optimization, analytics, and data engineering best practices for high-performance analytical workloads.","Sam-06060\u002Fjarvis-assistant","Sam-06060","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F208600848?v=4","01M6CRKT7C7FXD21V6HXF293S0","Samson Frank Ganta","01M6CRKTBEKYFD8YX5Z1GT99EE",6,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fcc-skill-clickhouse-io\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":26,"taxonomyTypeDesc":27},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44],"clickhouse","cc skill clickhouse io","clickhouse io","cc","performance","analytics","best practices","database","data engineering","engineering","Optimization","data","query optimization","community","patterns","query","2026-11-10 10:12:30",false,"---\nname: cc-skill-clickhouse-io\ndescription: \"ClickHouse database patterns, query optimization, analytics, and data engineering best practices for high-performance analytical workloads.\"\nrisk: unknown\nsource: community\ndate_added: \"2026-02-27\"\n---\n\n# ClickHouse Analytics Patterns\n\nClickHouse-specific patterns for high-performance analytics and data engineering.\n\n## Overview\n\nClickHouse is a column-oriented database management system (DBMS) for online analytical processing (OLAP). It's optimized for fast analytical queries on large datasets.\n\n**Key Features:**\n- Column-oriented storage\n- Data compression\n- Parallel query execution\n- Distributed queries\n- Real-time analytics\n\n## Table Design Patterns\n\n### MergeTree Engine (Most Common)\n\n```sql\nCREATE TABLE markets_analytics (\n    date Date,\n    market_id String,\n    market_name String,\n    volume UInt64,\n    trades UInt32,\n    unique_traders UInt32,\n    avg_trade_size Float64,\n    created_at DateTime\n) ENGINE = MergeTree()\nPARTITION BY toYYYYMM(date)\nORDER BY (date, market_id)\nSETTINGS index_granularity = 8192;\n```\n\n### ReplacingMergeTree (Deduplication)\n\n```sql\n-- For data that may have duplicates (e.g., from multiple sources)\nCREATE TABLE user_events (\n    event_id String,\n    user_id String,\n    event_type String,\n    timestamp DateTime,\n    properties String\n) ENGINE = ReplacingMergeTree()\nPARTITION BY toYYYYMM(timestamp)\nORDER BY (user_id, event_id, timestamp)\nPRIMARY KEY (user_id, event_id);\n```\n\n### AggregatingMergeTree (Pre-aggregation)\n\n```sql\n-- For maintaining aggregated metrics\nCREATE TABLE market_stats_hourly (\n    hour DateTime,\n    market_id String,\n    total_volume AggregateFunction(sum, UInt64),\n    total_trades AggregateFunction(count, UInt32),\n    unique_users AggregateFunction(uniq, String)\n) ENGINE = AggregatingMergeTree()\nPARTITION BY toYYYYMM(hour)\nORDER BY (hour, market_id);\n\n-- Query aggregated data\nSELECT\n    hour,\n    market_id,\n    sumMerge(total_volume) AS volume,\n    countMerge(total_trades) AS trades,\n    uniqMerge(unique_users) AS users\nFROM market_stats_hourly\nWHERE hour >= toStartOfHour(now() - INTERVAL 24 HOUR)\nGROUP BY hour, market_id\nORDER BY hour DESC;\n```\n\n## Query Optimization Patterns\n\n### Efficient Filtering\n\n```sql\n-- ✅ GOOD: Use indexed columns first\nSELECT *\nFROM markets_analytics\nWHERE date >= '2025-01-01'\n  AND market_id = 'market-123'\n  AND volume > 1000\nORDER BY date DESC\nLIMIT 100;\n\n-- ❌ BAD: Filter on non-indexed columns first\nSELECT *\nFROM markets_analytics\nWHERE volume > 1000\n  AND market_name LIKE '%election%'\n  AND date >= '2025-01-01';\n```\n\n### Aggregations\n\n```sql\n-- ✅ GOOD: Use ClickHouse-specific aggregation functions\nSELECT\n    toStartOfDay(created_at) AS day,\n    market_id,\n    sum(volume) AS total_volume,\n    count() AS total_trades,\n    uniq(trader_id) AS unique_traders,\n    avg(trade_size) AS avg_size\nFROM trades\nWHERE created_at >= today() - INTERVAL 7 DAY\nGROUP BY day, market_id\nORDER BY day DESC, total_volume DESC;\n\n-- ✅ Use quantile for percentiles (more efficient than percentile)\nSELECT\n    quantile(0.50)(trade_size) AS median,\n    quantile(0.95)(trade_size) AS p95,\n    quantile(0.99)(trade_size) AS p99\nFROM trades\nWHERE created_at >= now() - INTERVAL 1 HOUR;\n```\n\n### Window Functions\n\n```sql\n-- Calculate running totals\nSELECT\n    date,\n    market_id,\n    volume,\n    sum(volume) OVER (\n        PARTITION BY market_id\n        ORDER BY date\n        ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW\n    ) AS cumulative_volume\nFROM markets_analytics\nWHERE date >= today() - INTERVAL 30 DAY\nORDER BY market_id, date;\n```\n\n## Data Insertion Patterns\n\n### Bulk Insert (Recommended)\n\n```typescript\nimport { ClickHouse } from 'clickhouse'\n\nconst clickhouse = new ClickHouse({\n  url: process.env.CLICKHOUSE_URL,\n  port: 8123,\n  basicAuth: {\n    username: process.env.CLICKHOUSE_USER,\n    password: process.env.CLICKHOUSE_PASSWORD\n  }\n})\n\n\u002F\u002F ✅ Batch insert (efficient)\nasync function bulkInsertTrades(trades: Trade[]) {\n  const values = trades.map(trade => `(\n    '${trade.id}',\n    '${trade.market_id}',\n    '${trade.user_id}',\n    ${trade.amount},\n    '${trade.timestamp.toISOString()}'\n  )`).join(',')\n\n  await clickhouse.query(`\n    INSERT INTO trades (id, market_id, user_id, amount, timestamp)\n    VALUES ${values}\n  `).toPromise()\n}\n\n\u002F\u002F ❌ Individual inserts (slow)\nasync function insertTrade(trade: Trade) {\n  \u002F\u002F Don't do this in a loop!\n  await clickhouse.query(`\n    INSERT INTO trades VALUES ('${trade.id}', ...)\n  `).toPromise()\n}\n```\n\n### Streaming Insert\n\n```typescript\n\u002F\u002F For continuous data ingestion\nimport { createWriteStream } from 'fs'\nimport { pipeline } from 'stream\u002Fpromises'\n\nasync function streamInserts() {\n  const stream = clickhouse.insert('trades').stream()\n\n  for await (const batch of dataSource) {\n    stream.write(batch)\n  }\n\n  await stream.end()\n}\n```\n\n## Materialized Views\n\n### Real-time Aggregations\n\n```sql\n-- Create materialized view for hourly stats\nCREATE MATERIALIZED VIEW market_stats_hourly_mv\nTO market_stats_hourly\nAS SELECT\n    toStartOfHour(timestamp) AS hour,\n    market_id,\n    sumState(amount) AS total_volume,\n    countState() AS total_trades,\n    uniqState(user_id) AS unique_users\nFROM trades\nGROUP BY hour, market_id;\n\n-- Query the materialized view\nSELECT\n    hour,\n    market_id,\n    sumMerge(total_volume) AS volume,\n    countMerge(total_trades) AS trades,\n    uniqMerge(unique_users) AS users\nFROM market_stats_hourly\nWHERE hour >= now() - INTERVAL 24 HOUR\nGROUP BY hour, market_id;\n```\n\n## Performance Monitoring\n\n### Query Performance\n\n```sql\n-- Check slow queries\nSELECT\n    query_id,\n    user,\n    query,\n    query_duration_ms,\n    read_rows,\n    read_bytes,\n    memory_usage\nFROM system.query_log\nWHERE type = 'QueryFinish'\n  AND query_duration_ms > 1000\n  AND event_time >= now() - INTERVAL 1 HOUR\nORDER BY query_duration_ms DESC\nLIMIT 10;\n```\n\n### Table Statistics\n\n```sql\n-- Check table sizes\nSELECT\n    database,\n    table,\n    formatReadableSize(sum(bytes)) AS size,\n    sum(rows) AS rows,\n    max(modification_time) AS latest_modification\nFROM system.parts\nWHERE active\nGROUP BY database, table\nORDER BY sum(bytes) DESC;\n```\n\n## Common Analytics Queries\n\n### Time Series Analysis\n\n```sql\n-- Daily active users\nSELECT\n    toDate(timestamp) AS date,\n    uniq(user_id) AS daily_active_users\nFROM events\nWHERE timestamp >= today() - INTERVAL 30 DAY\nGROUP BY date\nORDER BY date;\n\n-- Retention analysis\nSELECT\n    signup_date,\n    countIf(days_since_signup = 0) AS day_0,\n    countIf(days_since_signup = 1) AS day_1,\n    countIf(days_since_signup = 7) AS day_7,\n    countIf(days_since_signup = 30) AS day_30\nFROM (\n    SELECT\n        user_id,\n        min(toDate(timestamp)) AS signup_date,\n        toDate(timestamp) AS activity_date,\n        dateDiff('day', signup_date, activity_date) AS days_since_signup\n    FROM events\n    GROUP BY user_id, activity_date\n)\nGROUP BY signup_date\nORDER BY signup_date DESC;\n```\n\n### Funnel Analysis\n\n```sql\n-- Conversion funnel\nSELECT\n    countIf(step = 'viewed_market') AS viewed,\n    countIf(step = 'clicked_trade') AS clicked,\n    countIf(step = 'completed_trade') AS completed,\n    round(clicked \u002F viewed * 100, 2) AS view_to_click_rate,\n    round(completed \u002F clicked * 100, 2) AS click_to_completion_rate\nFROM (\n    SELECT\n        user_id,\n        session_id,\n        event_type AS step\n    FROM events\n    WHERE event_date = today()\n)\nGROUP BY session_id;\n```\n\n### Cohort Analysis\n\n```sql\n-- User cohorts by signup month\nSELECT\n    toStartOfMonth(signup_date) AS cohort,\n    toStartOfMonth(activity_date) AS month,\n    dateDiff('month', cohort, month) AS months_since_signup,\n    count(DISTINCT user_id) AS active_users\nFROM (\n    SELECT\n        user_id,\n        min(toDate(timestamp)) OVER (PARTITION BY user_id) AS signup_date,\n        toDate(timestamp) AS activity_date\n    FROM events\n)\nGROUP BY cohort, month, months_since_signup\nORDER BY cohort, months_since_signup;\n```\n\n## Data Pipeline Patterns\n\n### ETL Pattern\n\n```typescript\n\u002F\u002F Extract, Transform, Load\nasync function etlPipeline() {\n  \u002F\u002F 1. Extract from source\n  const rawData = await extractFromPostgres()\n\n  \u002F\u002F 2. Transform\n  const transformed = rawData.map(row => ({\n    date: new Date(row.created_at).toISOString().split('T')[0],\n    market_id: row.market_slug,\n    volume: parseFloat(row.total_volume),\n    trades: parseInt(row.trade_count)\n  }))\n\n  \u002F\u002F 3. Load to ClickHouse\n  await bulkInsertToClickHouse(transformed)\n}\n\n\u002F\u002F Run periodically\nsetInterval(etlPipeline, 60 * 60 * 1000)  \u002F\u002F Every hour\n```\n\n### Change Data Capture (CDC)\n\n```typescript\n\u002F\u002F Listen to PostgreSQL changes and sync to ClickHouse\nimport { Client } from 'pg'\n\nconst pgClient = new Client({ connectionString: process.env.DATABASE_URL })\n\npgClient.query('LISTEN market_updates')\n\npgClient.on('notification', async (msg) => {\n  const update = JSON.parse(msg.payload)\n\n  await clickhouse.insert('market_updates', [\n    {\n      market_id: update.id,\n      event_type: update.operation,  \u002F\u002F INSERT, UPDATE, DELETE\n      timestamp: new Date(),\n      data: JSON.stringify(update.new_data)\n    }\n  ])\n})\n```\n\n## Best Practices\n\n### 1. Partitioning Strategy\n- Partition by time (usually month or day)\n- Avoid too many partitions (performance impact)\n- Use DATE type for partition key\n\n### 2. Ordering Key\n- Put most frequently filtered columns first\n- Consider cardinality (high cardinality first)\n- Order impacts compression\n\n### 3. Data Types\n- Use smallest appropriate type (UInt32 vs UInt64)\n- Use LowCardinality for repeated strings\n- Use Enum for categorical data\n\n### 4. Avoid\n- SELECT * (specify columns)\n- FINAL (merge data before query instead)\n- Too many JOINs (denormalize for analytics)\n- Small frequent inserts (batch instead)\n\n### 5. Monitoring\n- Track query performance\n- Monitor disk usage\n- Check merge operations\n- Review slow query log\n\n**Remember**: ClickHouse excels at analytical workloads. Design tables for your query patterns, batch inserts, and leverage materialized views for real-time aggregations.\n\n## When to Use\nThis skill is applicable to execute the workflow or actions described in the overview.\n","# ClickHouse Analytics Patterns\n\nClickHouse-specific patterns for high-performance analytics and data engineering.\n\n## Overview\n\nClickHouse is a column-oriented database management system (DBMS) for online analytical processing (OLAP). It's optimized for fast analytical queries on large datasets.\n\n**Key Features:**\n- Column-oriented storage\n- Data compression\n- Parallel query execution\n- Distributed queries\n- Real-time analytics\n\n## Table Design Patterns\n\n### MergeTree Engine (Most Common)\n\n```sql\nCREATE TABLE markets_analytics (\n    date Date,\n    market_id String,\n    market_name String,\n    volume UInt64,\n    trades UInt32,\n    unique_traders UInt32,\n    avg_trade_size Float64,\n    created_at DateTime\n) ENGINE = MergeTree()\nPARTITION BY toYYYYMM(date)\nORDER BY (date, market_id)\nSETTINGS index_granularity = 8192;\n```\n\n### ReplacingMergeTree (Deduplication)\n\n```sql\n-- For data that may have duplicates (e.g., from multiple sources)\nCREATE TABLE user_events (\n    event_id String,\n    user_id String,\n    event_type String,\n    timestamp DateTime,\n    properties String\n) ENGINE = ReplacingMergeTree()\nPARTITION BY toYYYYMM(timestamp)\nORDER BY (user_id, event_id, timestamp)\nPRIMARY KEY (user_id, event_id);\n```\n\n### AggregatingMergeTree (Pre-aggregation)\n\n```sql\n-- For maintaining aggregated metrics\nCREATE TABLE market_stats_hourly (\n    hour DateTime,\n    market_id String,\n    total_volume AggregateFunction(sum, UInt64),\n    total_trades AggregateFunction(count, UInt32),\n    unique_users AggregateFunction(uniq, String)\n) ENGINE = AggregatingMergeTree()\nPARTITION BY toYYYYMM(hour)\nORDER BY (hour, market_id);\n\n-- Query aggregated data\nSELECT\n    hour,\n    market_id,\n    sumMerge(total_volume) AS volume,\n    countMerge(total_trades) AS trades,\n    uniqMerge(unique_users) AS users\nFROM market_stats_hourly\nWHERE hour >= toStartOfHour(now() - INTERVAL 24 HOUR)\nGROUP BY hour, market_id\nORDER BY hour DESC;\n```\n\n## Query Optimization Patterns\n\n### Efficient Filtering\n\n```sql\n-- ✅ GOOD: Use indexed columns first\nSELECT *\nFROM markets_analytics\nWHERE date >= '2025-01-01'\n  AND market_id = 'market-123'\n  AND volume > 1000\nORDER BY date DESC\nLIMIT 100;\n\n-- ❌ BAD: Filter on non-indexed columns first\nSELECT *\nFROM markets_analytics\nWHERE volume > 1000\n  AND market_name LIKE '%election%'\n  AND date >= '2025-01-01';\n```\n\n### Aggregations\n\n```sql\n-- ✅ GOOD: Use ClickHouse-specific aggregation functions\nSELECT\n    toStartOfDay(created_at) AS day,\n    market_id,\n    sum(volume) AS total_volume,\n    count() AS total_trades,\n    uniq(trader_id) AS unique_traders,\n    avg(trade_size) AS avg_size\nFROM trades\nWHERE created_at >= today() - INTERVAL 7 DAY\nGROUP BY day, market_id\nORDER BY day DESC, total_volume DESC;\n\n-- ✅ Use quantile for percentiles (more efficient than percentile)\nSELECT\n    quantile(0.50)(trade_size) AS median,\n    quantile(0.95)(trade_size) AS p95,\n    quantile(0.99)(trade_size) AS p99\nFROM trades\nWHERE created_at >= now() - INTERVAL 1 HOUR;\n```\n\n### Window Functions\n\n```sql\n-- Calculate running totals\nSELECT\n    date,\n    market_id,\n    volume,\n    sum(volume) OVER (\n        PARTITION BY market_id\n        ORDER BY date\n        ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW\n    ) AS cumulative_volume\nFROM markets_analytics\nWHERE date >= today() - INTERVAL 30 DAY\nORDER BY market_id, date;\n```\n\n## Data Insertion Patterns\n\n### Bulk Insert (Recommended)\n\n```typescript\nimport { ClickHouse } from 'clickhouse'\n\nconst clickhouse = new ClickHouse({\n  url: process.env.CLICKHOUSE_URL,\n  port: 8123,\n  basicAuth: {\n    username: process.env.CLICKHOUSE_USER,\n    password: process.env.CLICKHOUSE_PASSWORD\n  }\n})\n\n\u002F\u002F ✅ Batch insert (efficient)\nasync function bulkInsertTrades(trades: Trade[]) {\n  const values = trades.map(trade => `(\n    '${trade.id}',\n    '${trade.market_id}',\n    '${trade.user_id}',\n    ${trade.amount},\n    '${trade.timestamp.toISOString()}'\n  )`).join(',')\n\n  await clickhouse.query(`\n    INSERT INTO trades (id, market_id, user_id, amount, timestamp)\n    VALUES ${values}\n  `).toPromise()\n}\n\n\u002F\u002F ❌ Individual inserts (slow)\nasync function insertTrade(trade: Trade) {\n  \u002F\u002F Don't do this in a loop!\n  await clickhouse.query(`\n    INSERT INTO trades VALUES ('${trade.id}', ...)\n  `).toPromise()\n}\n```\n\n### Streaming Insert\n\n```typescript\n\u002F\u002F For continuous data ingestion\nimport { createWriteStream } from 'fs'\nimport { pipeline } from 'stream\u002Fpromises'\n\nasync function streamInserts() {\n  const stream = clickhouse.insert('trades').stream()\n\n  for await (const batch of dataSource) {\n    stream.write(batch)\n  }\n\n  await stream.end()\n}\n```\n\n## Materialized Views\n\n### Real-time Aggregations\n\n```sql\n-- Create materialized view for hourly stats\nCREATE MATERIALIZED VIEW market_stats_hourly_mv\nTO market_stats_hourly\nAS SELECT\n    toStartOfHour(timestamp) AS hour,\n    market_id,\n    sumState(amount) AS total_volume,\n    countState() AS total_trades,\n    uniqState(user_id) AS unique_users\nFROM trades\nGROUP BY hour, market_id;\n\n-- Query the materialized view\nSELECT\n    hour,\n    market_id,\n    sumMerge(total_volume) AS volume,\n    countMerge(total_trades) AS trades,\n    uniqMerge(unique_users) AS users\nFROM market_stats_hourly\nWHERE hour >= now() - INTERVAL 24 HOUR\nGROUP BY hour, market_id;\n```\n\n## Performance Monitoring\n\n### Query Performance\n\n```sql\n-- Check slow queries\nSELECT\n    query_id,\n    user,\n    query,\n    query_duration_ms,\n    read_rows,\n    read_bytes,\n    memory_usage\nFROM system.query_log\nWHERE type = 'QueryFinish'\n  AND query_duration_ms > 1000\n  AND event_time >= now() - INTERVAL 1 HOUR\nORDER BY query_duration_ms DESC\nLIMIT 10;\n```\n\n### Table Statistics\n\n```sql\n-- Check table sizes\nSELECT\n    database,\n    table,\n    formatReadableSize(sum(bytes)) AS size,\n    sum(rows) AS rows,\n    max(modification_time) AS latest_modification\nFROM system.parts\nWHERE active\nGROUP BY database, table\nORDER BY sum(bytes) DESC;\n```\n\n## Common Analytics Queries\n\n### Time Series Analysis\n\n```sql\n-- Daily active users\nSELECT\n    toDate(timestamp) AS date,\n    uniq(user_id) AS daily_active_users\nFROM events\nWHERE timestamp >= today() - INTERVAL 30 DAY\nGROUP BY date\nORDER BY date;\n\n-- Retention analysis\nSELECT\n    signup_date,\n    countIf(days_since_signup = 0) AS day_0,\n    countIf(days_since_signup = 1) AS day_1,\n    countIf(days_since_signup = 7) AS day_7,\n    countIf(days_since_signup = 30) AS day_30\nFROM (\n    SELECT\n        user_id,\n        min(toDate(timestamp)) AS signup_date,\n        toDate(timestamp) AS activity_date,\n        dateDiff('day', signup_date, activity_date) AS days_since_signup\n    FROM events\n    GROUP BY user_id, activity_date\n)\nGROUP BY signup_date\nORDER BY signup_date DESC;\n```\n\n### Funnel Analysis\n\n```sql\n-- Conversion funnel\nSELECT\n    countIf(step = 'viewed_market') AS viewed,\n    countIf(step = 'clicked_trade') AS clicked,\n    countIf(step = 'completed_trade') AS completed,\n    round(clicked \u002F viewed * 100, 2) AS view_to_click_rate,\n    round(completed \u002F clicked * 100, 2) AS click_to_completion_rate\nFROM (\n    SELECT\n        user_id,\n        session_id,\n        event_type AS step\n    FROM events\n    WHERE event_date = today()\n)\nGROUP BY session_id;\n```\n\n### Cohort Analysis\n\n```sql\n-- User cohorts by signup month\nSELECT\n    toStartOfMonth(signup_date) AS cohort,\n    toStartOfMonth(activity_date) AS month,\n    dateDiff('month', cohort, month) AS months_since_signup,\n    count(DISTINCT user_id) AS active_users\nFROM (\n    SELECT\n        user_id,\n        min(toDate(timestamp)) OVER (PARTITION BY user_id) AS signup_date,\n        toDate(timestamp) AS activity_date\n    FROM events\n)\nGROUP BY cohort, month, months_since_signup\nORDER BY cohort, months_since_signup;\n```\n\n## Data Pipeline Patterns\n\n### ETL Pattern\n\n```typescript\n\u002F\u002F Extract, Transform, Load\nasync function etlPipeline() {\n  \u002F\u002F 1. Extract from source\n  const rawData = await extractFromPostgres()\n\n  \u002F\u002F 2. Transform\n  const transformed = rawData.map(row => ({\n    date: new Date(row.created_at).toISOString().split('T')[0],\n    market_id: row.market_slug,\n    volume: parseFloat(row.total_volume),\n    trades: parseInt(row.trade_count)\n  }))\n\n  \u002F\u002F 3. Load to ClickHouse\n  await bulkInsertToClickHouse(transformed)\n}\n\n\u002F\u002F Run periodically\nsetInterval(etlPipeline, 60 * 60 * 1000)  \u002F\u002F Every hour\n```\n\n### Change Data Capture (CDC)\n\n```typescript\n\u002F\u002F Listen to PostgreSQL changes and sync to ClickHouse\nimport { Client } from 'pg'\n\nconst pgClient = new Client({ connectionString: process.env.DATABASE_URL })\n\npgClient.query('LISTEN market_updates')\n\npgClient.on('notification', async (msg) => {\n  const update = JSON.parse(msg.payload)\n\n  await clickhouse.insert('market_updates', [\n    {\n      market_id: update.id,\n      event_type: update.operation,  \u002F\u002F INSERT, UPDATE, DELETE\n      timestamp: new Date(),\n      data: JSON.stringify(update.new_data)\n    }\n  ])\n})\n```\n\n## Best Practices\n\n### 1. Partitioning Strategy\n- Partition by time (usually month or day)\n- Avoid too many partitions (performance impact)\n- Use DATE type for partition key\n\n### 2. Ordering Key\n- Put most frequently filtered columns first\n- Consider cardinality (high cardinality first)\n- Order impacts compression\n\n### 3. Data Types\n- Use smallest appropriate type (UInt32 vs UInt64)\n- Use LowCardinality for repeated strings\n- Use Enum for categorical data\n\n### 4. Avoid\n- SELECT * (specify columns)\n- FINAL (merge data before query instead)\n- Too many JOINs (denormalize for analytics)\n- Small frequent inserts (batch instead)\n\n### 5. Monitoring\n- Track query performance\n- Monitor disk usage\n- Check merge operations\n- Review slow query log\n\n**Remember**: ClickHouse excels at analytical workloads. Design tables for your query patterns, batch inserts, and leverage materialized views for real-time aggregations.\n\n## When to Use\nThis skill is applicable to execute the workflow or actions described in the overview.\n",{"name":5,"risk":50,"source":42,"date_added":51,"description":7},"unknown","2026-02-27","main","a50a3aa6b39fb0e1b7db88bef1bfc9c93606e126","modules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fcc-skill-clickhouse-io",0,"https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant","https:\u002F\u002Fgithub.com\u002FSam-06060","Mobile Application Developer | Kotlin | Flutter, KMP, Swfit, Android,Swift UI | Building at Saar It Resources | Creator of Jarvis (AI Assistant)","2026-06-23 05:13:38",{"source":8,"skillName":5,"commands":61,"prompt":71,"zipAvailable":72},[62,65,68],{"tool":63,"command":64},"npx","npx skills add Sam-06060\u002Fjarvis-assistant --skill cc-skill-clickhouse-io",{"tool":66,"command":67},"bunx","bunx skills add Sam-06060\u002Fjarvis-assistant --skill cc-skill-clickhouse-io",{"tool":69,"command":70},"pnpm","pnpm dlx skills add Sam-06060\u002Fjarvis-assistant --skill cc-skill-clickhouse-io","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fcc-skill-clickhouse-io\u002FSKILL.md",true,[74,80,86,90,94,98,102,106,110,114,118,122,126,130,134,138,142,146,150,154,160,164],{"path":75,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":78,"sourceHtmlUrl":79},"assets\u002Fevals\u002Fhyper-research-cases.jsonl","ASSET","资产文件",24012,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Fassets\u002Fevals\u002Fhyper-research-cases.jsonl",{"path":81,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":84,"sourceHtmlUrl":85},"references\u002Fchannel-selection.ko.md","REFERENCE","参考文档",5866,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Freferences\u002Fchannel-selection.ko.md",{"path":87,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":88,"sourceHtmlUrl":89},"references\u002Fchannel-selection.md",5359,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Freferences\u002Fchannel-selection.md",{"path":91,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":92,"sourceHtmlUrl":93},"references\u002Fjournal-format.ko.md",7741,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Freferences\u002Fjournal-format.ko.md",{"path":95,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":96,"sourceHtmlUrl":97},"references\u002Fjournal-format.md",7170,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Freferences\u002Fjournal-format.md",{"path":99,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":100,"sourceHtmlUrl":101},"references\u002Flatex-report.ko.md",7219,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Freferences\u002Flatex-report.ko.md",{"path":103,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":104,"sourceHtmlUrl":105},"references\u002Flatex-report.md",6510,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Freferences\u002Flatex-report.md",{"path":107,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":108,"sourceHtmlUrl":109},"references\u002Freport-template.ko.md",5396,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Freferences\u002Freport-template.ko.md",{"path":111,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":112,"sourceHtmlUrl":113},"references\u002Freport-template.md",5127,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Freferences\u002Freport-template.md",{"path":115,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":116,"sourceHtmlUrl":117},"rules\u002Fclaim-verification.ko.md",10047,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fclaim-verification.ko.md",{"path":119,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":120,"sourceHtmlUrl":121},"rules\u002Fclaim-verification.md",8679,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fclaim-verification.md",{"path":123,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":124,"sourceHtmlUrl":125},"rules\u002Fexpansion-and-convergence.ko.md",6826,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fexpansion-and-convergence.ko.md",{"path":127,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":128,"sourceHtmlUrl":129},"rules\u002Fexpansion-and-convergence.md",5558,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fexpansion-and-convergence.md",{"path":131,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":132,"sourceHtmlUrl":133},"rules\u002Fparallel-research.ko.md",6389,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fparallel-research.ko.md",{"path":135,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":136,"sourceHtmlUrl":137},"rules\u002Fparallel-research.md",5465,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fparallel-research.md",{"path":139,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":140,"sourceHtmlUrl":141},"rules\u002Fresearch-workflow.ko.md",8464,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fresearch-workflow.ko.md",{"path":143,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":144,"sourceHtmlUrl":145},"rules\u002Fresearch-workflow.md",7337,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fresearch-workflow.md",{"path":147,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":148,"sourceHtmlUrl":149},"rules\u002Fvalidation.ko.md",11835,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fvalidation.ko.md",{"path":151,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":152,"sourceHtmlUrl":153},"rules\u002Fvalidation.md",10322,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Frules\u002Fvalidation.md",{"path":155,"resourceType":156,"resourceTypeDesc":157,"sizeBytes":158,"sourceHtmlUrl":159},"scripts\u002Freport-state.sh","SCRIPT","脚本",16894,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Fscripts\u002Freport-state.sh",{"path":161,"resourceType":156,"resourceTypeDesc":157,"sizeBytes":162,"sourceHtmlUrl":163},"scripts\u002Fvalidate-report.sh",16550,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002Fscripts\u002Fvalidate-report.sh",{"path":165,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":166,"sourceHtmlUrl":167},"SKILL.ko.md",20715,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Falpoxdev\u002Fhypercore\u002Fskills\u002Fhyper-research\u002FSKILL.ko.md",[169,180,195,222,241,268],{"skillNo":170,"name":171,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":172,"categories":173,"keywords":177,"publishedAt":45},"01M6GDRVS6FXX3ESH0C9DW01C0","test_skill","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fscratch\u002Ftest_skill\u002FSKILL.md",[174],{"categoryNo":175,"categoryName":176,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",[178,179],"Test Skill","test",{"skillNo":181,"name":182,"nameZh":6,"description":183,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":184,"categories":185,"keywords":186,"publishedAt":45},"01M6GDRQ40HX22MA6XWC31RHHK","zustand-store-ts","Create Zustand stores following established patterns with proper TypeScript types and middleware.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzustand-store-ts\u002FSKILL.md",[],[187,188,189,190,191,192,43,42,193,194],"zustand store ts","zustand","store","with","types","middleware","create","typescript",{"skillNo":196,"name":197,"nameZh":6,"description":198,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":199,"categories":200,"keywords":204,"publishedAt":45},"01M6GDRKEA0WE670SPFWX5GKCN","zoom-automation","Automate Zoom meeting creation, management, recordings, webinars, and participant tracking via Rube MCP (Composio). Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzoom-automation\u002FSKILL.md",[201],{"categoryNo":202,"categoryName":203,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[205,206,207,208,209,210,211,212,213,214,215,216,217,218,42,219,220,221],"automation","zoom automation","zoom","first","Tools","search","MCP","via","meeting","automate","critical","creation","webinars","composio","recordings","management","tracking",{"skillNo":223,"name":224,"nameZh":6,"description":225,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":226,"categories":227,"keywords":229,"publishedAt":45},"01M6GDRFS1452PC71JQ38S24HR","zoho-crm-automation","Automate Zoho CRM tasks via Rube MCP (Composio): create\u002Fupdate records, search contacts, manage leads, and convert leads. Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzoho-crm-automation\u002FSKILL.md",[228],{"categoryNo":202,"categoryName":203,"taxonomyType":21,"taxonomyTypeDesc":22},[230,205,231,232,233,234,235,42,210,236,218,237,215,214,209,238,193,239,211,240],"crm automation","zoho crm","crm","zoho crm automation","zoho","leads","contacts","records","update","tasks","manage",{"skillNo":242,"name":243,"nameZh":6,"description":244,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":245,"categories":246,"keywords":250,"publishedAt":45},"01M6GDRCKQBAVTGHX5XBH26KPZ","zod-validation-expert","Expert in Zod — TypeScript-first schema validation. Covers parsing, custom errors, refinements, type inference, and integration with React Hook Form, Next.js, and tRPC.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzod-validation-expert\u002FSKILL.md",[247],{"categoryNo":248,"categoryName":249,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000009","前端开发",[251,252,253,254,255,194,256,257,258,259,260,261,190,262,263,264,265,42,266,267],"expert","zod","zod validation expert","validation","react","safe","parsing","errors","schema validation","react hook form","hook","trpc","form","type","next","schema","Next.js",{"skillNo":269,"name":270,"nameZh":6,"description":271,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":272,"categories":273,"keywords":275,"publishedAt":45},"01M6GDR658B08K4MY0RWD4A6EH","zeroize-audit","Detects missing zeroization of sensitive data in source code and identifies zeroization removed by compiler optimizations, with assembly-level analysis, and control-flow verification. Use for auditing C\u002FC++\u002FRust code handling secrets, keys, passwords, or other sensitive data.","https:\u002F\u002Fgithub.com\u002FSam-06060\u002Fjarvis-assistant\u002Fblob\u002Fmain\u002Fmodules\u002Fagent_skills\u002Fawesome-skills\u002Fplaybooks\u002Fzeroize-audit\u002FSKILL.md",[274],{"categoryNo":202,"categoryName":203,"taxonomyType":21,"taxonomyTypeDesc":22},[276,277,278,279,280,281,282,283,42,284,285,286,287,288,40,289,290,291,292,211],"zeroize audit","audit","write","passwords","get","project","c++","activate","assembly","read","Bash","secrets","analysis","flow","verification","code","rust"]