rag-architect
Use when the user asks to design a RAG pipeline, choose a chunking strategy or embedding model, pick a vector database, or evaluate retrieval quality (precision@k, recall@k, NDCG). Examples: 'design a RAG system for our docs', 'what chunk size should I use for this corpus', 'evaluate my retriever against ground truth'. NOT for general LLM cost tuning (use llm-cost-optimizer) or agent loops over retrieval (use agenthub).
- License 许可
- MIT
- 最近更新
- 2026-11-04
- Stars
- ★ 2
- Forks
- 0
01 前言 Front Matter
- name
- rag-architect
- description
- Use when the user asks to design a RAG pipeline, choose a chunking strategy or embedding model, pick a vector database, or evaluate retrieval quality (precision@k, recall@k, NDCG). Examples: 'design a RAG system for our docs', 'what chunk size should I use for this corpus', 'evaluate my retriever against ground truth'. NOT for general LLM cost tuning (use llm-cost-optimizer) or agent loops over retrieval (use agenthub).
02 资源文件 6
- 参考文档chunking_optimizer.py.txt28.8 KB
- 参考文档rag_pipeline_designer.py.txt27.6 KB
- 参考文档references/chunking_strategies_comparison.md9.9 KB
- 参考文档references/embedding_model_benchmark.md12.4 KB
- 参考文档references/rag_evaluation_framework.md14.8 KB
- 参考文档retrieval_evaluator.py.txt23.0 KB
03 SKILL.md 正文
RAG Architect
Design, tune, and evaluate production RAG pipelines with three deterministic tools. Run the tools against the actual corpus and requirements — do not pick chunk sizes or databases by intuition.
Hard rules
- Never present model names or vendor prices as current facts. Embedding models and vector-DB pricing rot in months. Recommend a tier (see table below), name a current-generation candidate, and tell the user to verify against the provider's live pricing page.
- Every design ends with an evaluation run. A RAG design without
retrieval_evaluator.pynumbers is a hypothesis, not a deliverable. - Chunking is corpus-driven. Run
chunking_optimizer.pyon the real documents before choosing a strategy.
Embedding model tiers (pattern, not price list)
| Tier | Current-generation examples (verify before use) | When |
|---|---|---|
| Fast / self-hosted | all-MiniLM-L6-v2, bge-small |
Cost-sensitive, small scale, real-time |
| Balanced open | all-mpnet-base-v2, bge-large, e5-large |
Quality without API dependency |
| Quality API | text-embedding-3-large, voyage-3-large |
Accuracy-priority general retrieval |
| Code | voyage-code-3, CodeBERT-family |
Code search corpora |
Pricing discipline: build the cost model with a placeholder table — columns model | $/1M tokens (verify) | dims | as-of date — and have the user fill in live numbers. Same for vector DBs (Pinecone/Weaviate/Qdrant/Chroma/pgvector): the selection criteria (managed vs self-hosted, scale, filtering, existing Postgres) are durable; the dollar figures are not.
Workflow
All paths relative to this skill folder. Outputs chain: corpus analysis → design → evaluation.
1. Analyze the corpus and pick chunking
python3 chunking_optimizer.py /path/to/docs --extensions .md .txt -o chunking.json
Emits chunking.json with corpus_info, per-strategy strategy_results, a recommendation, and sample_chunks. Use recommendation.strategy and its config; show the user 2-3 sample_chunks so they can sanity-check boundaries.
2. Design the pipeline from requirements
Write a requirements JSON with these keys (all required): document_types[], document_count, avg_document_size (chars), queries_per_day, query_patterns[], latency_requirement, budget_monthly, accuracy_priority (0-1), cost_priority (0-1), maintenance_complexity.
python3 rag_pipeline_designer.py requirements.json -o design.json
Emits design.json with chunking, embedding, vector_db, retrieval, reranking, evaluation, total_cost, architecture_diagram (mermaid), and config_templates. Present the diagram; label every cost_monthly figure as an estimate to verify (rule 1).
3. Evaluate retrieval quality
Prepare queries.json (list of {id, text} or {"queries": [...]}) and ground_truth.json ({query_id: [relevant_doc_ids]}), then:
python3 retrieval_evaluator.py queries.json /path/to/docs ground_truth.json --k-values 3 5 10 -o eval.json
Reports precision@k, recall@k, MRR, NDCG@k, plus poor_precision_examples / poor_recall_examples for failure analysis.
4. Verification loop
The design is done only when:
eval.jsonmeets targets — typical floors: precision@5 ≥ 0.8, recall@10 ≥ 0.85 (set per use case with the user).- If below target: inspect the poor-example lists, then change one variable (chunking strategy → re-run step 1; embedding tier; add reranking; hybrid retrieval) and re-run step 3. Repeat.
- Every recommended model/price in the deliverable carries a "verify current pricing/model availability" note with an as-of date.
References
references/chunking_strategies_comparison.md— strategy trade-offs the optimizer implementsreferences/embedding_model_benchmark.md— benchmark methodology (dated snapshot; staleness warning at top)references/rag_evaluation_framework.md— metric definitions (faithfulness, relevance, precision/recall/NDCG)
04 同仓库更多 Skills
webapp-testing
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
youtube-downloader
Download YouTube videos with customizable quality and format options. Use this skill when the user asks to download, save, or grab YouTube videos. Supports various quality settings (best, 1080p, 720p, 480p, 360p), multiple formats (mp4, webm, mkv), and audio-only downloads as MP3.
theme-factory
Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.
template-skill
Replace with description of the skill and when Claude should use it.
tailored-resume-generator
Analyzes job descriptions and generates tailored resumes that highlight relevant experience, skills, and achievements to maximize interview chances
slack-gif-creator
Toolkit for creating animated GIFs optimized for Slack, with validators for size constraints and composable animation primitives. This skill applies when users request animated GIFs or emoji animations for Slack from descriptions like "make me a GIF for Slack of X doing Y".