AGENT SKILL

vector-store-architect

Architect-level vector database design — index type and recall/latency trade-offs, filtering and tenancy, metadata schema, sharding and replication, updates and deletes, and the operational difference between pgvector, Qdrant, Weaviate, and managed services. Use when vectors need a home beyond a prototype.

License 许可
MIT
最近更新
2026-11-13
Stars
★ 0
Forks
0

01 前言 Front Matter

name
vector-store-architect
license
MIT
metadata
{ "crusor-path": "ai-ml-architect/retrieval-architect/vector-store-architect", "crusor-domain": "ai-ml-architect", "crusor-origin": "original", "crusor-sources": "github.com/pgvector/pgvector; qdrant.tech/documentation; weaviate.io/developers/weaviate; docs.pinecone.io", "crusor-triggers": "vector database, pgvector, qdrant, weaviate, pinecone, hnsw, ivf, ann index, similarity search, vector index, vector search, irrelevant results, retrieval quality", "crusor-verified-on": "2026-10-30" }
description
Architect-level vector database design — index type and recall/latency trade-offs, filtering and tenancy, metadata schema, sharding and replication, updates and deletes, and the operational difference between pgvector, Qdrant, Weaviate, and managed services. Use when vectors need a home beyond a prototype.
compatibility
pgvector, Qdrant, Weaviate, Milvus, Pinecone, OpenSearch

03 SKILL.md 正文

Vector Store Architect

A vector store is an index with a consistency model. Most failures are not recall problems — they are filtering, tenancy, and delete semantics.

When to use

  • More than a few hundred thousand vectors, or strict latency targets.
  • Metadata filtering combined with similarity search.
  • Multi-tenant retrieval with isolation requirements.

When NOT to use

  • Small corpus where Postgres with pgvector is already deployed and sufficient.
  • Pure keyword search — that is a text index.

Decision 1 — Engine

Engine Fit
pgvector already on Postgres, moderate scale, transactional consistency
Qdrant filtering-heavy, self-hosted, strong payload model
Weaviate hybrid search and modules out of the box
Milvus very large scale, dedicated ops
Pinecone managed, low ops, higher cost at scale
OpenSearch/Elasticsearch existing search platform, hybrid needs
  • Introducing a new datastore means a new backup, auth, and monitoring story. Pay it only for a real capability gap.
  • Estimate: vectors × dimensions × 4 bytes × (1 + index overhead) — then compare to available RAM.

Decision 2 — Index

Index Trade-off
flat/exact perfect recall, slow at scale
HNSW fast queries, memory-hungry, high build cost
IVF-PQ memory efficient, recall loss, tuning required
disk-based (DiskANN) large corpora with acceptable latency
  • Always measure recall@k against exact search on a sample; "it feels fast" is not recall.
  • Tune m/ef_construction/nprobe against latency budget, then freeze the parameters with the index version.
  • Rebuilding an index is an operational event: plan capacity and a maintenance window.

Decision 3 — Schema and filtering

  • Metadata carries: tenant, document id, section, timestamps, permissions, embedding model version.
  • Filter inside the vector search (pre-filter), not after retrieval — post-filter silently returns fewer than k results.
  • Pre-filtering with high-selectivity filters can be slow on HNSW; check the engine's guidance.
  • Store the text or a pointer to it; you will need to re-embed and to show snippets.

Decision 4 — Tenancy

Model Isolation
one collection, tenant in payload weakest, filter must be enforced everywhere
collection per tenant stronger, many collections to manage
namespace/partition per tenant balance, engine-dependent
database/cluster per tenant strongest, most expensive
  • Enforce the tenant filter in the data-access layer, never by trusting the caller's query.
  • Test cross-tenant leakage as a first-class test case.

Decision 5 — Writes and deletes

  • Upsert by content hash + document id; deletion by document id must remove all its chunks.
  • Deletes are often deferred/async — verify the vector is actually gone before claiming deletion.
  • Versioning: keep the embedding model id on each vector to make migration and audit possible.
  • Snapshot/backup strategy: a rebuilt index from source text is preferable to backing up vectors, if ingestion is fast.

Failure modes at scale

  • Post-filter dropping results: "top 5" often returns 2.
  • One collection, tenant filter forgotten in a new code path; data leaks.
  • HNSW index in RAM exceeding the node size; OOM under load.
  • Deleted documents still returned weeks later because deletes were queued.
  • Recall degraded after an index rebuild with default parameters.

Verification receipts

  1. Engine choice with the capability that justified a new datastore.
  2. Recall@k measured against exact search on a sample.
  3. Filter selectivity test: filtered queries still return k results.
  4. Cross-tenant test proving isolation.
  5. Delete verification: document removed from every chunk and result.

04 同仓库更多 Skills

SriAkhilSJ
SriAkhilSJ/PulseAIRepo

interaction-patterns-architect

Architect-level interaction design in code — choosing established patterns over invented ones, feedback and latency behaviour, destructive-action safeguards, navigation and overlay conventions, and the state model behind every interactive surface. Use when a screen has to behave, not just look.

★ 0
SriAkhilSJ
SriAkhilSJ/PulseAIRepo

design-system-ops-architect

Architect-level operation of a design system in code — token pipelines and versioning, component API and variant policy, the catalogue as the contract, adoption metrics, and the governance that stops a system from forking into three. Use when a component library must survive several teams.

★ 0
SriAkhilSJ
SriAkhilSJ/PulseAIRepo

design-handoff-architect

Architect-level design handoff — what must be specified before implementation starts, tokens over raw values, responsive and content-limit contracts, state matrices, and the review loop that catches drift between design and code. Use when a design is about to become code.

★ 0
SriAkhilSJ
SriAkhilSJ/PulseAIRepo

content-microcopy-architect

Architect-level UI content — where strings live, voice and terminology consistency, error and empty-state copy as part of the interface, translation-ready authoring, and the content model a product needs before it multiplies screens. Use when the words in the interface matter as much as the pixels.

★ 0
SriAkhilSJ
SriAkhilSJ/PulseAIRepo

ui-ux-implementation-architect

Architect-level direction for turning a design into working UI — the handoff contract, component ownership versus screen-level layout, interaction and state conventions, design QA gates, and the review habits that keep an implementation faithful without frozen pixels. Use before any specific UI implementation skill.

★ 0
SriAkhilSJ
SriAkhilSJ/PulseAIRepo

test-strategy-architect

Architect-level test strategy — allocating assertions across unit, integration, contract and end-to-end layers, designing for testability through dependency boundaries, fixture and data strategy, and the flake and coverage policies that keep a suite trustworthy. Use when defining how a codebase is tested.

★ 0