[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6FGFBCW2DRBBR9BCN69X7DP":3,"skill-same:01M6FGFBCW2DRBBR9BCN69X7DP":118},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":51,"commitSha":52,"skillDirectory":53,"forksCount":54,"repositoryHtmlUrl":55,"repositoryHomepageUrl":56,"ownerHtmlUrl":57,"ownerBlogUrl":6,"ownerDescription":58,"updatedAt":59,"install":60,"resources":73},"01M6FGFBCW2DRBBR9BCN69X7DP","database-sharding",null,"Database sharding for PostgreSQL\u002FMySQL with hash\u002Frange\u002Fdirectory strategies. Use for horizontal scaling, multi-tenant isolation, billions of records, or encountering wrong shard keys, hotspots, cross-shard transactions, rebalancing issues.","ComeOnOliver\u002Fskillshub","ComeOnOliver","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F43215937?v=4","01M6E5JNFECJAAVFE0W7SMWWD2","Haoran Shu","01M6E5JNNDQS7AJZDYFQ8TPZCK",67,"MIT","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"database sharding","Sharding","database","router","isolation","Saga Pattern","scaling","strategies","pattern","scalability","selection","directory","virtual","transactions","shard","postgresql","mysql","distributed database","issues","commit","2026-11-10 10:12:30",false,"---\nname: database-sharding\ndescription: Database sharding for PostgreSQL\u002FMySQL with hash\u002Frange\u002Fdirectory strategies. Use for horizontal scaling, multi-tenant isolation, billions of records, or encountering wrong shard keys, hotspots, cross-shard transactions, rebalancing issues.\nlicense: MIT\nmetadata:\n  keywords: \"database sharding, horizontal partitioning, shard key, consistent hashing, hash sharding, range sharding, directory sharding, cross-shard queries, shard rebalancing, scatter-gather pattern, multi-tenant sharding, shard router, distributed database, database scalability, shard migration, hotspot shards, shard key selection, cross-shard aggregation, two-phase commit, saga pattern, virtual shards\"\n---\n\n# database-sharding\n\nComprehensive database sharding patterns for horizontal scaling with hash, range, and directory-based strategies.\n\n---\n\n## Quick Start (10 Minutes)\n\n**Step 1**: Choose sharding strategy from templates:\n```bash\n# Hash-based (even distribution)\ncat templates\u002Fhash-router.ts\n\n# Range-based (time-series data)\ncat templates\u002Frange-router.ts\n\n# Directory-based (multi-tenancy)\ncat templates\u002Fdirectory-router.ts\n```\n\n**Step 2**: Select shard key criteria:\n- ✅ **High cardinality** (millions of unique values)\n- ✅ **Even distribution** (no single value > 5%)\n- ✅ **Immutable** (never changes)\n- ✅ **Query alignment** (in 80%+ of WHERE clauses)\n\n**Step 3**: Implement router:\n```typescript\nimport { HashRouter } from '.\u002Fhash-router';\n\nconst router = new HashRouter([\n  { id: 'shard_0', connection: { host: 'db0.example.com' } },\n  { id: 'shard_1', connection: { host: 'db1.example.com' } },\n  { id: 'shard_2', connection: { host: 'db2.example.com' } },\n  { id: 'shard_3', connection: { host: 'db3.example.com' } },\n]);\n\n\u002F\u002F Query single shard\nconst user = await router.query('user_123', 'SELECT * FROM users WHERE id = $1', ['user_123']);\n```\n\n---\n\n## Critical Rules\n\n### ✓ Always Do\n\n| Rule | Reason |\n|------|--------|\n| **Include shard key in queries** | Avoid scanning all shards (100x slower) |\n| **Monitor shard distribution** | Detect hotspots before they cause outages |\n| **Plan for rebalancing upfront** | Cannot easily add shards later |\n| **Choose immutable shard key** | Changing key = data migration nightmare |\n| **Test distribution with production data** | Synthetic data hides real hotspots |\n| **Denormalize for data locality** | Keep related data on same shard |\n\n### ✗ Never Do\n\n| Anti-Pattern | Why It's Bad |\n|--------------|--------------|\n| **Sequential ID with range sharding** | Latest shard gets all writes (hotspot) |\n| **Timestamp as shard key** | Recent shard overwhelmed |\n| **Cross-shard transactions without 2PC** | Data corruption, inconsistency |\n| **Simple modulo without consistent hashing** | Cannot add shards without full re-shard |\n| **Nullable shard key** | Special NULL handling creates hotspots |\n| **No shard routing layer** | Hardcoded shards = cannot rebalance |\n\n---\n\n## Top 7 Critical Errors\n\n### Error 1: Wrong Shard Key Choice (Hotspots)\n**Symptom**: One shard receives 80%+ of traffic\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Low cardinality (status field)\nshard_key = order.status; \u002F\u002F 90% are 'pending' → shard_0 overloaded\n\n\u002F\u002F ✅ Good: High cardinality (user_id)\nshard_key = order.user_id; \u002F\u002F Millions of users, even distribution\n```\n\n### Error 2: Missing Shard Key in Queries\n**Symptom**: Queries scan ALL shards (extremely slow)\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: No shard key\nSELECT * FROM orders WHERE status = 'shipped'; \u002F\u002F Scans all 100 shards!\n\n\u002F\u002F ✅ Good: Include shard key\nSELECT * FROM orders WHERE user_id = ? AND status = 'shipped'; \u002F\u002F Targets 1 shard\n```\n\n### Error 3: Sequential IDs with Range Sharding\n**Symptom**: Latest shard gets all writes\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Range sharding with auto-increment\n\u002F\u002F Shard 0: 1-1M, Shard 1: 1M-2M, Shard 2: 2M+ → All new writes to Shard 2!\n\n\u002F\u002F ✅ Good: Hash-based sharding\nconst shardId = hash(id) % shardCount; \u002F\u002F Even distribution\n```\n\n### Error 4: No Rebalancing Strategy\n**Symptom**: Stuck with initial shard count, cannot scale\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Simple modulo\nconst shardId = hash(key) % shardCount; \u002F\u002F Adding 5th shard breaks ALL keys\n\n\u002F\u002F ✅ Good: Consistent hashing\nconst ring = new ConsistentHashRing(shards);\nconst shardId = ring.getNode(key); \u002F\u002F Only ~25% of keys move when adding shard\n```\n\n### Error 5: Cross-Shard Transactions\n**Symptom**: Data inconsistency, partial writes\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Cross-shard transaction (will corrupt)\nBEGIN;\nUPDATE shard_1.accounts SET balance = balance - 100 WHERE id = 'A';\nUPDATE shard_2.accounts SET balance = balance + 100 WHERE id = 'B';\nCOMMIT; \u002F\u002F If shard_2 fails, shard_1 already committed!\n\n\u002F\u002F ✅ Good: Two-Phase Commit or Saga pattern\nconst txn = new TwoPhaseCommitTransaction();\ntxn.addOperation(shard_1, 'UPDATE accounts SET balance = balance - 100 WHERE id = ?', ['A']);\ntxn.addOperation(shard_2, 'UPDATE accounts SET balance = balance + 100 WHERE id = ?', ['B']);\nawait txn.execute(); \u002F\u002F Atomic across shards\n```\n\n### Error 6: Mutable Shard Key\n**Symptom**: Records move shards, causing duplicates\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Shard by country (user relocates)\nshard_key = user.country; \u002F\u002F User moves US → CA, now in different shard!\n\n\u002F\u002F ✅ Good: Shard by immutable user_id\nshard_key = user.id; \u002F\u002F Never changes\n```\n\n### Error 7: No Monitoring\n**Symptom**: Silent hotspots, sudden performance degradation\n**Fix**:\n```typescript\n\u002F\u002F ✅ Required metrics\n- Per-shard record counts (should be within 20%)\n- Query distribution (no shard > 40% of queries)\n- Storage per shard (alert at 80%)\n- Latency p99 per shard\n```\n\n**Load** `references\u002Ferror-catalog.md` for all 10 errors with detailed fixes.\n\n---\n\n## Sharding Strategies\n\n| Strategy | Best For | Pros | Cons |\n|----------|----------|------|------|\n| **Hash** | User data, even load critical | No hotspots, predictable | Range queries scatter |\n| **Range** | Time-series, logs, append-only | Range queries efficient, archival | Recent shard hotspot |\n| **Directory** | Multi-tenancy, complex routing | Flexible, easy rebalancing | Lookup overhead, SPOF |\n\n**Load** `references\u002Fsharding-strategies.md` for detailed comparisons with production examples (Instagram, Discord, Salesforce).\n\n---\n\n## Shard Key Selection Criteria\n\n| Criterion | Importance | Check Method |\n|-----------|------------|--------------|\n| **High cardinality** | Critical | `COUNT(DISTINCT shard_key)` > shard_count × 100 |\n| **Even distribution** | Critical | No value > 5% of total |\n| **Immutable** | Critical | Value never changes |\n| **Query alignment** | High | 80%+ queries include it |\n| **Data locality** | Medium | Related records together |\n\n**Decision Tree**:\n- User-focused app → `user_id`\n- Multi-tenant SaaS → `tenant_id`\n- Time-series\u002Flogs → `timestamp` (range sharding)\n- Product catalog → `product_id`\n\n**Load** `references\u002Fshard-key-selection.md` for comprehensive decision trees and testing strategies.\n\n---\n\n## Configuration Summary\n\n### Hash-Based Router\n\n```typescript\nimport { HashRouter } from '.\u002Ftemplates\u002Fhash-router';\n\nconst router = new HashRouter([\n  { id: 'shard_0', connection: { \u002F* PostgreSQL config *\u002F } },\n  { id: 'shard_1', connection: { \u002F* PostgreSQL config *\u002F } },\n]);\n\n\u002F\u002F Automatically routes to correct shard\nconst user = await router.query('user_123', 'SELECT * FROM users WHERE id = $1', ['user_123']);\n```\n\n### Range-Based Router\n\n```typescript\nimport { RangeRouter } from '.\u002Ftemplates\u002Frange-router';\n\nconst router = new RangeRouter(shardConfigs, [\n  { start: Date.parse('2024-01-01'), end: Date.parse('2024-04-01'), shardId: 'shard_q1' },\n  { start: Date.parse('2024-04-01'), end: Date.parse('2024-07-01'), shardId: 'shard_q2' },\n  { start: Date.parse('2024-07-01'), end: Infinity, shardId: 'shard_q3' },\n]);\n\n\u002F\u002F Range queries target specific shards\nconst janEvents = await router.queryRange(\n  Date.parse('2024-01-01'),\n  Date.parse('2024-02-01'),\n  'SELECT * FROM events WHERE created_at BETWEEN $1 AND $2'\n);\n```\n\n### Directory-Based Router\n\n```typescript\nimport { DirectoryRouter } from '.\u002Ftemplates\u002Fdirectory-router';\n\nconst router = new DirectoryRouter(directoryDBConfig, shardConfigs);\n\n\u002F\u002F Assign tenant to specific shard\nawait router.assignShard('tenant_acme', 'shard_enterprise');\n\n\u002F\u002F Route automatically\nconst users = await router.query('tenant_acme', 'SELECT * FROM users');\n```\n\n---\n\n## When to Load References\n\n### Choosing Strategy\n**Load** `references\u002Fsharding-strategies.md` when:\n- Deciding between hash, range, directory\n- Need production examples (Instagram, Discord)\n- Planning hybrid approaches\n\n### Selecting Shard Key\n**Load** `references\u002Fshard-key-selection.md` when:\n- Choosing shard key for new project\n- Evaluating existing shard key\n- Testing distribution with production data\n\n### Implementation\n**Load** `references\u002Fimplementation-patterns.md` when:\n- Building shard router from scratch\n- Implementing consistent hashing\n- Need transaction handling (2PC, Saga)\n- Setting up monitoring\u002Fmetrics\n\n### Cross-Shard Operations\n**Load** `references\u002Fcross-shard-queries.md` when:\n- Need to aggregate across shards (COUNT, SUM, AVG)\n- Implementing cross-shard joins\n- Building pagination across shards\n- Optimizing scatter-gather patterns\n\n### Rebalancing\n**Load** `references\u002Frebalancing-guide.md` when:\n- Adding new shards\n- Migrating data between shards\n- Planning zero-downtime migrations\n- Balancing uneven load\n\n### Error Prevention\n**Load** `references\u002Ferror-catalog.md` when:\n- Troubleshooting performance issues\n- Reviewing shard architecture\n- All 10 documented errors with fixes\n\n---\n\n## Complete Setup Checklist\n\n**Before Sharding**:\n- [ ] Tested shard key distribution with production data\n- [ ] Shard key in 80%+ of queries\n- [ ] Monitoring infrastructure ready\n- [ ] Rebalancing strategy planned\n\n**Router Implementation**:\n- [ ] Shard routing layer (not hardcoded shards)\n- [ ] Connection pooling per shard\n- [ ] Error handling and retries\n- [ ] Metrics collection (queries\u002Fshard, latency)\n\n**Shard Configuration**:\n- [ ] 4-8 shards initially (room to grow)\n- [ ] Consistent hashing or virtual shards\n- [ ] Replicas per shard (HA)\n- [ ] Backup strategy per shard\n\n**Application Changes**:\n- [ ] All queries include shard key\n- [ ] Cross-shard joins eliminated (denormalized)\n- [ ] Transaction boundaries respected\n- [ ] Connection pooling configured\n\n---\n\n## Production Example\n\n**Before** (Single database overwhelmed):\n```typescript\n\u002F\u002F Single PostgreSQL instance\nconst db = new Pool({ host: 'db.example.com' });\n\n\u002F\u002F All 10M users on one server\nconst users = await db.query('SELECT * FROM users WHERE status = $1', ['active']);\n\u002F\u002F Query time: 5000ms (slow!)\n\u002F\u002F DB CPU: 95%\n\u002F\u002F Disk: 500GB, growing\n```\n\n**After** (Sharded across 8 servers):\n```typescript\n\u002F\u002F Hash-based sharding with 8 shards\nconst router = new HashRouter([\n  { id: 'shard_0', connection: { host: 'db0.example.com' } },\n  { id: 'shard_1', connection: { host: 'db1.example.com' } },\n  \u002F\u002F ... 6 more shards\n]);\n\n\u002F\u002F Query single user (targets 1 shard)\nconst user = await router.query('user_123', 'SELECT * FROM users WHERE id = $1', ['user_123']);\n\u002F\u002F Query time: 10ms (500x faster!)\n\n\u002F\u002F Query all shards (scatter-gather)\nconst allActive = await router.queryAll('SELECT * FROM users WHERE status = $1', ['active']);\n\u002F\u002F Query time: 800ms (parallelized across 8 shards, 6x faster than single)\n\n\u002F\u002F Result: Each shard handles ~1.25M users\n\u002F\u002F DB CPU per shard: 20%\n\u002F\u002F Disk per shard: 65GB\n\u002F\u002F Can scale to 16 shards easily (consistent hashing)\n```\n\n---\n\n## Known Issues Prevention\n\nAll 10 documented errors prevented:\n1. ✅ Wrong shard key (hotspots) → Test distribution first\n2. ✅ Missing shard key in queries → Code review, linting\n3. ✅ Cross-shard transactions → Use 2PC or Saga pattern\n4. ✅ Sequential ID hotspots → Use hash-based sharding\n5. ✅ No rebalancing strategy → Consistent hashing from day 1\n6. ✅ Timestamp sharding hotspots → Hybrid hash+range approach\n7. ✅ Mutable shard key → Choose immutable keys (user_id)\n8. ✅ No routing layer → Abstract with router from start\n9. ✅ No monitoring → Track per-shard metrics\n10. ✅ Weak hash function → Use MD5, MurmurHash3, xxHash\n\n**See**: `references\u002Ferror-catalog.md` for detailed fixes\n\n---\n\n## Resources\n\n**Templates**:\n- `templates\u002Fhash-router.ts` - Hash-based sharding\n- `templates\u002Frange-router.ts` - Range-based sharding\n- `templates\u002Fdirectory-router.ts` - Directory-based sharding\n- `templates\u002Fcross-shard-aggregation.ts` - Aggregation patterns\n\n**References**:\n- `references\u002Fsharding-strategies.md` - Strategy comparison\n- `references\u002Fshard-key-selection.md` - Key selection guide\n- `references\u002Fimplementation-patterns.md` - Router implementations\n- `references\u002Fcross-shard-queries.md` - Query patterns\n- `references\u002Frebalancing-guide.md` - Migration strategies\n- `references\u002Ferror-catalog.md` - All 10 errors documented\n\n**Production Examples**:\n- Instagram: Range sharding for media\n- Discord: Hash sharding for messages\n- Salesforce: Directory sharding for orgs\n\n---\n\n**Production-tested** | **10 errors prevented** | **MIT License**\n","# database-sharding\n\nComprehensive database sharding patterns for horizontal scaling with hash, range, and directory-based strategies.\n\n---\n\n## Quick Start (10 Minutes)\n\n**Step 1**: Choose sharding strategy from templates:\n```bash\n# Hash-based (even distribution)\ncat templates\u002Fhash-router.ts\n\n# Range-based (time-series data)\ncat templates\u002Frange-router.ts\n\n# Directory-based (multi-tenancy)\ncat templates\u002Fdirectory-router.ts\n```\n\n**Step 2**: Select shard key criteria:\n- ✅ **High cardinality** (millions of unique values)\n- ✅ **Even distribution** (no single value > 5%)\n- ✅ **Immutable** (never changes)\n- ✅ **Query alignment** (in 80%+ of WHERE clauses)\n\n**Step 3**: Implement router:\n```typescript\nimport { HashRouter } from '.\u002Fhash-router';\n\nconst router = new HashRouter([\n  { id: 'shard_0', connection: { host: 'db0.example.com' } },\n  { id: 'shard_1', connection: { host: 'db1.example.com' } },\n  { id: 'shard_2', connection: { host: 'db2.example.com' } },\n  { id: 'shard_3', connection: { host: 'db3.example.com' } },\n]);\n\n\u002F\u002F Query single shard\nconst user = await router.query('user_123', 'SELECT * FROM users WHERE id = $1', ['user_123']);\n```\n\n---\n\n## Critical Rules\n\n### ✓ Always Do\n\n| Rule | Reason |\n|------|--------|\n| **Include shard key in queries** | Avoid scanning all shards (100x slower) |\n| **Monitor shard distribution** | Detect hotspots before they cause outages |\n| **Plan for rebalancing upfront** | Cannot easily add shards later |\n| **Choose immutable shard key** | Changing key = data migration nightmare |\n| **Test distribution with production data** | Synthetic data hides real hotspots |\n| **Denormalize for data locality** | Keep related data on same shard |\n\n### ✗ Never Do\n\n| Anti-Pattern | Why It's Bad |\n|--------------|--------------|\n| **Sequential ID with range sharding** | Latest shard gets all writes (hotspot) |\n| **Timestamp as shard key** | Recent shard overwhelmed |\n| **Cross-shard transactions without 2PC** | Data corruption, inconsistency |\n| **Simple modulo without consistent hashing** | Cannot add shards without full re-shard |\n| **Nullable shard key** | Special NULL handling creates hotspots |\n| **No shard routing layer** | Hardcoded shards = cannot rebalance |\n\n---\n\n## Top 7 Critical Errors\n\n### Error 1: Wrong Shard Key Choice (Hotspots)\n**Symptom**: One shard receives 80%+ of traffic\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Low cardinality (status field)\nshard_key = order.status; \u002F\u002F 90% are 'pending' → shard_0 overloaded\n\n\u002F\u002F ✅ Good: High cardinality (user_id)\nshard_key = order.user_id; \u002F\u002F Millions of users, even distribution\n```\n\n### Error 2: Missing Shard Key in Queries\n**Symptom**: Queries scan ALL shards (extremely slow)\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: No shard key\nSELECT * FROM orders WHERE status = 'shipped'; \u002F\u002F Scans all 100 shards!\n\n\u002F\u002F ✅ Good: Include shard key\nSELECT * FROM orders WHERE user_id = ? AND status = 'shipped'; \u002F\u002F Targets 1 shard\n```\n\n### Error 3: Sequential IDs with Range Sharding\n**Symptom**: Latest shard gets all writes\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Range sharding with auto-increment\n\u002F\u002F Shard 0: 1-1M, Shard 1: 1M-2M, Shard 2: 2M+ → All new writes to Shard 2!\n\n\u002F\u002F ✅ Good: Hash-based sharding\nconst shardId = hash(id) % shardCount; \u002F\u002F Even distribution\n```\n\n### Error 4: No Rebalancing Strategy\n**Symptom**: Stuck with initial shard count, cannot scale\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Simple modulo\nconst shardId = hash(key) % shardCount; \u002F\u002F Adding 5th shard breaks ALL keys\n\n\u002F\u002F ✅ Good: Consistent hashing\nconst ring = new ConsistentHashRing(shards);\nconst shardId = ring.getNode(key); \u002F\u002F Only ~25% of keys move when adding shard\n```\n\n### Error 5: Cross-Shard Transactions\n**Symptom**: Data inconsistency, partial writes\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Cross-shard transaction (will corrupt)\nBEGIN;\nUPDATE shard_1.accounts SET balance = balance - 100 WHERE id = 'A';\nUPDATE shard_2.accounts SET balance = balance + 100 WHERE id = 'B';\nCOMMIT; \u002F\u002F If shard_2 fails, shard_1 already committed!\n\n\u002F\u002F ✅ Good: Two-Phase Commit or Saga pattern\nconst txn = new TwoPhaseCommitTransaction();\ntxn.addOperation(shard_1, 'UPDATE accounts SET balance = balance - 100 WHERE id = ?', ['A']);\ntxn.addOperation(shard_2, 'UPDATE accounts SET balance = balance + 100 WHERE id = ?', ['B']);\nawait txn.execute(); \u002F\u002F Atomic across shards\n```\n\n### Error 6: Mutable Shard Key\n**Symptom**: Records move shards, causing duplicates\n**Fix**:\n```typescript\n\u002F\u002F ❌ Bad: Shard by country (user relocates)\nshard_key = user.country; \u002F\u002F User moves US → CA, now in different shard!\n\n\u002F\u002F ✅ Good: Shard by immutable user_id\nshard_key = user.id; \u002F\u002F Never changes\n```\n\n### Error 7: No Monitoring\n**Symptom**: Silent hotspots, sudden performance degradation\n**Fix**:\n```typescript\n\u002F\u002F ✅ Required metrics\n- Per-shard record counts (should be within 20%)\n- Query distribution (no shard > 40% of queries)\n- Storage per shard (alert at 80%)\n- Latency p99 per shard\n```\n\n**Load** `references\u002Ferror-catalog.md` for all 10 errors with detailed fixes.\n\n---\n\n## Sharding Strategies\n\n| Strategy | Best For | Pros | Cons |\n|----------|----------|------|------|\n| **Hash** | User data, even load critical | No hotspots, predictable | Range queries scatter |\n| **Range** | Time-series, logs, append-only | Range queries efficient, archival | Recent shard hotspot |\n| **Directory** | Multi-tenancy, complex routing | Flexible, easy rebalancing | Lookup overhead, SPOF |\n\n**Load** `references\u002Fsharding-strategies.md` for detailed comparisons with production examples (Instagram, Discord, Salesforce).\n\n---\n\n## Shard Key Selection Criteria\n\n| Criterion | Importance | Check Method |\n|-----------|------------|--------------|\n| **High cardinality** | Critical | `COUNT(DISTINCT shard_key)` > shard_count × 100 |\n| **Even distribution** | Critical | No value > 5% of total |\n| **Immutable** | Critical | Value never changes |\n| **Query alignment** | High | 80%+ queries include it |\n| **Data locality** | Medium | Related records together |\n\n**Decision Tree**:\n- User-focused app → `user_id`\n- Multi-tenant SaaS → `tenant_id`\n- Time-series\u002Flogs → `timestamp` (range sharding)\n- Product catalog → `product_id`\n\n**Load** `references\u002Fshard-key-selection.md` for comprehensive decision trees and testing strategies.\n\n---\n\n## Configuration Summary\n\n### Hash-Based Router\n\n```typescript\nimport { HashRouter } from '.\u002Ftemplates\u002Fhash-router';\n\nconst router = new HashRouter([\n  { id: 'shard_0', connection: { \u002F* PostgreSQL config *\u002F } },\n  { id: 'shard_1', connection: { \u002F* PostgreSQL config *\u002F } },\n]);\n\n\u002F\u002F Automatically routes to correct shard\nconst user = await router.query('user_123', 'SELECT * FROM users WHERE id = $1', ['user_123']);\n```\n\n### Range-Based Router\n\n```typescript\nimport { RangeRouter } from '.\u002Ftemplates\u002Frange-router';\n\nconst router = new RangeRouter(shardConfigs, [\n  { start: Date.parse('2024-01-01'), end: Date.parse('2024-04-01'), shardId: 'shard_q1' },\n  { start: Date.parse('2024-04-01'), end: Date.parse('2024-07-01'), shardId: 'shard_q2' },\n  { start: Date.parse('2024-07-01'), end: Infinity, shardId: 'shard_q3' },\n]);\n\n\u002F\u002F Range queries target specific shards\nconst janEvents = await router.queryRange(\n  Date.parse('2024-01-01'),\n  Date.parse('2024-02-01'),\n  'SELECT * FROM events WHERE created_at BETWEEN $1 AND $2'\n);\n```\n\n### Directory-Based Router\n\n```typescript\nimport { DirectoryRouter } from '.\u002Ftemplates\u002Fdirectory-router';\n\nconst router = new DirectoryRouter(directoryDBConfig, shardConfigs);\n\n\u002F\u002F Assign tenant to specific shard\nawait router.assignShard('tenant_acme', 'shard_enterprise');\n\n\u002F\u002F Route automatically\nconst users = await router.query('tenant_acme', 'SELECT * FROM users');\n```\n\n---\n\n## When to Load References\n\n### Choosing Strategy\n**Load** `references\u002Fsharding-strategies.md` when:\n- Deciding between hash, range, directory\n- Need production examples (Instagram, Discord)\n- Planning hybrid approaches\n\n### Selecting Shard Key\n**Load** `references\u002Fshard-key-selection.md` when:\n- Choosing shard key for new project\n- Evaluating existing shard key\n- Testing distribution with production data\n\n### Implementation\n**Load** `references\u002Fimplementation-patterns.md` when:\n- Building shard router from scratch\n- Implementing consistent hashing\n- Need transaction handling (2PC, Saga)\n- Setting up monitoring\u002Fmetrics\n\n### Cross-Shard Operations\n**Load** `references\u002Fcross-shard-queries.md` when:\n- Need to aggregate across shards (COUNT, SUM, AVG)\n- Implementing cross-shard joins\n- Building pagination across shards\n- Optimizing scatter-gather patterns\n\n### Rebalancing\n**Load** `references\u002Frebalancing-guide.md` when:\n- Adding new shards\n- Migrating data between shards\n- Planning zero-downtime migrations\n- Balancing uneven load\n\n### Error Prevention\n**Load** `references\u002Ferror-catalog.md` when:\n- Troubleshooting performance issues\n- Reviewing shard architecture\n- All 10 documented errors with fixes\n\n---\n\n## Complete Setup Checklist\n\n**Before Sharding**:\n- [ ] Tested shard key distribution with production data\n- [ ] Shard key in 80%+ of queries\n- [ ] Monitoring infrastructure ready\n- [ ] Rebalancing strategy planned\n\n**Router Implementation**:\n- [ ] Shard routing layer (not hardcoded shards)\n- [ ] Connection pooling per shard\n- [ ] Error handling and retries\n- [ ] Metrics collection (queries\u002Fshard, latency)\n\n**Shard Configuration**:\n- [ ] 4-8 shards initially (room to grow)\n- [ ] Consistent hashing or virtual shards\n- [ ] Replicas per shard (HA)\n- [ ] Backup strategy per shard\n\n**Application Changes**:\n- [ ] All queries include shard key\n- [ ] Cross-shard joins eliminated (denormalized)\n- [ ] Transaction boundaries respected\n- [ ] Connection pooling configured\n\n---\n\n## Production Example\n\n**Before** (Single database overwhelmed):\n```typescript\n\u002F\u002F Single PostgreSQL instance\nconst db = new Pool({ host: 'db.example.com' });\n\n\u002F\u002F All 10M users on one server\nconst users = await db.query('SELECT * FROM users WHERE status = $1', ['active']);\n\u002F\u002F Query time: 5000ms (slow!)\n\u002F\u002F DB CPU: 95%\n\u002F\u002F Disk: 500GB, growing\n```\n\n**After** (Sharded across 8 servers):\n```typescript\n\u002F\u002F Hash-based sharding with 8 shards\nconst router = new HashRouter([\n  { id: 'shard_0', connection: { host: 'db0.example.com' } },\n  { id: 'shard_1', connection: { host: 'db1.example.com' } },\n  \u002F\u002F ... 6 more shards\n]);\n\n\u002F\u002F Query single user (targets 1 shard)\nconst user = await router.query('user_123', 'SELECT * FROM users WHERE id = $1', ['user_123']);\n\u002F\u002F Query time: 10ms (500x faster!)\n\n\u002F\u002F Query all shards (scatter-gather)\nconst allActive = await router.queryAll('SELECT * FROM users WHERE status = $1', ['active']);\n\u002F\u002F Query time: 800ms (parallelized across 8 shards, 6x faster than single)\n\n\u002F\u002F Result: Each shard handles ~1.25M users\n\u002F\u002F DB CPU per shard: 20%\n\u002F\u002F Disk per shard: 65GB\n\u002F\u002F Can scale to 16 shards easily (consistent hashing)\n```\n\n---\n\n## Known Issues Prevention\n\nAll 10 documented errors prevented:\n1. ✅ Wrong shard key (hotspots) → Test distribution first\n2. ✅ Missing shard key in queries → Code review, linting\n3. ✅ Cross-shard transactions → Use 2PC or Saga pattern\n4. ✅ Sequential ID hotspots → Use hash-based sharding\n5. ✅ No rebalancing strategy → Consistent hashing from day 1\n6. ✅ Timestamp sharding hotspots → Hybrid hash+range approach\n7. ✅ Mutable shard key → Choose immutable keys (user_id)\n8. ✅ No routing layer → Abstract with router from start\n9. ✅ No monitoring → Track per-shard metrics\n10. ✅ Weak hash function → Use MD5, MurmurHash3, xxHash\n\n**See**: `references\u002Ferror-catalog.md` for detailed fixes\n\n---\n\n## Resources\n\n**Templates**:\n- `templates\u002Fhash-router.ts` - Hash-based sharding\n- `templates\u002Frange-router.ts` - Range-based sharding\n- `templates\u002Fdirectory-router.ts` - Directory-based sharding\n- `templates\u002Fcross-shard-aggregation.ts` - Aggregation patterns\n\n**References**:\n- `references\u002Fsharding-strategies.md` - Strategy comparison\n- `references\u002Fshard-key-selection.md` - Key selection guide\n- `references\u002Fimplementation-patterns.md` - Router implementations\n- `references\u002Fcross-shard-queries.md` - Query patterns\n- `references\u002Frebalancing-guide.md` - Migration strategies\n- `references\u002Ferror-catalog.md` - All 10 errors documented\n\n**Production Examples**:\n- Instagram: Range sharding for media\n- Discord: Hash sharding for messages\n- Salesforce: Directory sharding for orgs\n\n---\n\n**Production-tested** | **10 errors prevented** | **MIT License**\n",{"name":5,"license":15,"metadata":49,"description":7},{"keywords":50},"database sharding, horizontal partitioning, shard key, consistent hashing, hash sharding, range sharding, directory sharding, cross-shard queries, shard rebalancing, scatter-gather pattern, multi-tenant sharding, shard router, distributed database, database scalability, shard migration, hotspot shards, shard key selection, cross-shard aggregation, two-phase commit, saga pattern, virtual shards","main","def8531e65114c0fca8fb8551c1871ee0eed705c","skills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding",24,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub","https:\u002F\u002Fskillshub.wtf","https:\u002F\u002Fgithub.com\u002FComeOnOliver","AI & Blockchain Engineer\r\n","2026-06-24 22:07:19",{"source":8,"skillName":5,"commands":61,"prompt":71,"zipAvailable":72},[62,65,68],{"tool":63,"command":64},"npx","npx skills add ComeOnOliver\u002Fskillshub --skill database-sharding",{"tool":66,"command":67},"bunx","bunx skills add ComeOnOliver\u002Fskillshub --skill database-sharding",{"tool":69,"command":70},"pnpm","pnpm dlx skills add ComeOnOliver\u002Fskillshub --skill database-sharding","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002FSKILL.md",true,[74,80,84,88,92,96,100,106,110,114],{"path":75,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":78,"sourceHtmlUrl":79},"references\u002Fcross-shard-queries.md","REFERENCE","参考文档",8451,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Freferences\u002Fcross-shard-queries.md",{"path":81,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Ferror-catalog.md",16134,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Freferences\u002Ferror-catalog.md",{"path":85,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":86,"sourceHtmlUrl":87},"references\u002Fimplementation-patterns.md",13100,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Freferences\u002Fimplementation-patterns.md",{"path":89,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":90,"sourceHtmlUrl":91},"references\u002Frebalancing-guide.md",8279,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Freferences\u002Frebalancing-guide.md",{"path":93,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":94,"sourceHtmlUrl":95},"references\u002Fshard-key-selection.md",14312,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Freferences\u002Fshard-key-selection.md",{"path":97,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":98,"sourceHtmlUrl":99},"references\u002Fsharding-strategies.md",13584,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Freferences\u002Fsharding-strategies.md",{"path":101,"resourceType":102,"resourceTypeDesc":103,"sizeBytes":104,"sourceHtmlUrl":105},"templates\u002Fcross-shard-aggregation.ts","SCRIPT","脚本",6265,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Ftemplates\u002Fcross-shard-aggregation.ts",{"path":107,"resourceType":102,"resourceTypeDesc":103,"sizeBytes":108,"sourceHtmlUrl":109},"templates\u002Fdirectory-router.ts",5015,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Ftemplates\u002Fdirectory-router.ts",{"path":111,"resourceType":102,"resourceTypeDesc":103,"sizeBytes":112,"sourceHtmlUrl":113},"templates\u002Fhash-router.ts",2537,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Ftemplates\u002Fhash-router.ts",{"path":115,"resourceType":102,"resourceTypeDesc":103,"sizeBytes":116,"sourceHtmlUrl":117},"templates\u002Frange-router.ts",3066,"https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fsecondsky\u002Fclaude-skills\u002Fdatabase-sharding\u002Ftemplates\u002Frange-router.ts",[119,131,141,150,156,163],{"skillNo":120,"name":121,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":122,"categories":123,"keywords":127,"publishedAt":44},"01M6FKJN13GMG4KY57J3QD8GY5","model-hierarchy-skill","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzscole\u002Fmodel-hierarchy-skill\u002Fmodel-hierarchy-skill\u002FSKILL.md",[124],{"categoryNo":125,"categoryName":126,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[128,129,130],"model","model hierarchy","model hierarchy skill",{"skillNo":132,"name":133,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":134,"categories":135,"keywords":137,"publishedAt":44},"01M6FKJJJ6GCAMR4PPPRD12M5X","zoom-mcp","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fzoom-mcp\u002FSKILL.md",[136],{"categoryNo":125,"categoryName":126,"taxonomyType":21,"taxonomyTypeDesc":22},[138,139,140],"MCP","zoom","zoom mcp",{"skillNo":142,"name":143,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":144,"categories":145,"keywords":146,"publishedAt":44},"01M6FKJE3KYT3KPEZMEEHYKA6K","zoom-apps-sdk","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fzoom-apps-sdk\u002FSKILL.md",[],[147,148,149,139],"sdk","zoom apps sdk","apps",{"skillNo":151,"name":152,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":155,"publishedAt":44},"01M6FKJ9EDH9Q5VWDF8HAJRSP0","windows","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwindows\u002FSKILL.md",[],[152],{"skillNo":157,"name":158,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":159,"categories":160,"keywords":162,"publishedAt":44},"01M6FKJ666BEEFC3AE9GJ72Q93","whiteboard","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwhiteboard\u002FSKILL.md",[161],{"categoryNo":125,"categoryName":126,"taxonomyType":21,"taxonomyTypeDesc":22},[158],{"skillNo":164,"name":165,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":166,"categories":167,"keywords":168,"publishedAt":44},"01M6FKJ21WFCDGTJQ2Q91T9DVW","websockets","https:\u002F\u002Fgithub.com\u002FComeOnOliver\u002Fskillshub\u002Fblob\u002Fmain\u002Fskills\u002Fzoom\u002Fskills\u002Fwebsockets\u002FSKILL.md",[],[165]]