vector-store-architect
Architect-level vector database design — index type and recall/latency trade-offs, filtering and tenancy, metadata schema, sharding and replication, updates and deletes, and the operational difference between pgvector, Qdrant, Weaviate, and managed services. Use when vectors need a home beyond a prototype.
- License 许可
- MIT
- 最近更新
- 2026-11-13
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- vector-store-architect
- license
- MIT
- metadata
- { "crusor-path": "ai-ml-architect/retrieval-architect/vector-store-architect", "crusor-domain": "ai-ml-architect", "crusor-origin": "original", "crusor-sources": "github.com/pgvector/pgvector; qdrant.tech/documentation; weaviate.io/developers/weaviate; docs.pinecone.io", "crusor-triggers": "vector database, pgvector, qdrant, weaviate, pinecone, hnsw, ivf, ann index, similarity search, vector index, vector search, irrelevant results, retrieval quality", "crusor-verified-on": "2026-10-30" }
- description
- Architect-level vector database design — index type and recall/latency trade-offs, filtering and tenancy, metadata schema, sharding and replication, updates and deletes, and the operational difference between pgvector, Qdrant, Weaviate, and managed services. Use when vectors need a home beyond a prototype.
- compatibility
- pgvector, Qdrant, Weaviate, Milvus, Pinecone, OpenSearch
03 SKILL.md 正文
Vector Store Architect
A vector store is an index with a consistency model. Most failures are not recall problems — they are filtering, tenancy, and delete semantics.
When to use
- More than a few hundred thousand vectors, or strict latency targets.
- Metadata filtering combined with similarity search.
- Multi-tenant retrieval with isolation requirements.
When NOT to use
- Small corpus where Postgres with pgvector is already deployed and sufficient.
- Pure keyword search — that is a text index.
Decision 1 — Engine
| Engine | Fit |
|---|---|
| pgvector | already on Postgres, moderate scale, transactional consistency |
| Qdrant | filtering-heavy, self-hosted, strong payload model |
| Weaviate | hybrid search and modules out of the box |
| Milvus | very large scale, dedicated ops |
| Pinecone | managed, low ops, higher cost at scale |
| OpenSearch/Elasticsearch | existing search platform, hybrid needs |
- Introducing a new datastore means a new backup, auth, and monitoring story. Pay it only for a real capability gap.
- Estimate: vectors × dimensions × 4 bytes × (1 + index overhead) — then compare to available RAM.
Decision 2 — Index
| Index | Trade-off |
|---|---|
| flat/exact | perfect recall, slow at scale |
| HNSW | fast queries, memory-hungry, high build cost |
| IVF-PQ | memory efficient, recall loss, tuning required |
| disk-based (DiskANN) | large corpora with acceptable latency |
- Always measure recall@k against exact search on a sample; "it feels fast" is not recall.
- Tune
m/ef_construction/nprobeagainst latency budget, then freeze the parameters with the index version. - Rebuilding an index is an operational event: plan capacity and a maintenance window.
Decision 3 — Schema and filtering
- Metadata carries: tenant, document id, section, timestamps, permissions, embedding model version.
- Filter inside the vector search (pre-filter), not after retrieval — post-filter silently returns fewer than k results.
- Pre-filtering with high-selectivity filters can be slow on HNSW; check the engine's guidance.
- Store the text or a pointer to it; you will need to re-embed and to show snippets.
Decision 4 — Tenancy
| Model | Isolation |
|---|---|
| one collection, tenant in payload | weakest, filter must be enforced everywhere |
| collection per tenant | stronger, many collections to manage |
| namespace/partition per tenant | balance, engine-dependent |
| database/cluster per tenant | strongest, most expensive |
- Enforce the tenant filter in the data-access layer, never by trusting the caller's query.
- Test cross-tenant leakage as a first-class test case.
Decision 5 — Writes and deletes
- Upsert by content hash + document id; deletion by document id must remove all its chunks.
- Deletes are often deferred/async — verify the vector is actually gone before claiming deletion.
- Versioning: keep the embedding model id on each vector to make migration and audit possible.
- Snapshot/backup strategy: a rebuilt index from source text is preferable to backing up vectors, if ingestion is fast.
Failure modes at scale
- Post-filter dropping results: "top 5" often returns 2.
- One collection, tenant filter forgotten in a new code path; data leaks.
- HNSW index in RAM exceeding the node size; OOM under load.
- Deleted documents still returned weeks later because deletes were queued.
- Recall degraded after an index rebuild with default parameters.
Verification receipts
- Engine choice with the capability that justified a new datastore.
- Recall@k measured against exact search on a sample.
- Filter selectivity test: filtered queries still return k results.
- Cross-tenant test proving isolation.
- Delete verification: document removed from every chunk and result.
04 同仓库更多 Skills
interaction-patterns-architect
Architect-level interaction design in code — choosing established patterns over invented ones, feedback and latency behaviour, destructive-action safeguards, navigation and overlay conventions, and the state model behind every interactive surface. Use when a screen has to behave, not just look.
design-system-ops-architect
Architect-level operation of a design system in code — token pipelines and versioning, component API and variant policy, the catalogue as the contract, adoption metrics, and the governance that stops a system from forking into three. Use when a component library must survive several teams.
design-handoff-architect
Architect-level design handoff — what must be specified before implementation starts, tokens over raw values, responsive and content-limit contracts, state matrices, and the review loop that catches drift between design and code. Use when a design is about to become code.
content-microcopy-architect
Architect-level UI content — where strings live, voice and terminology consistency, error and empty-state copy as part of the interface, translation-ready authoring, and the content model a product needs before it multiplies screens. Use when the words in the interface matter as much as the pixels.
ui-ux-implementation-architect
Architect-level direction for turning a design into working UI — the handoff contract, component ownership versus screen-level layout, interaction and state conventions, design QA gates, and the review habits that keep an implementation faithful without frozen pixels. Use before any specific UI implementation skill.
test-strategy-architect
Architect-level test strategy — allocating assertions across unit, integration, contract and end-to-end layers, designing for testability through dependency boundaries, fixture and data strategy, and the flake and coverage policies that keep a suite trustworthy. Use when defining how a codebase is tested.