rag-core
Build and troubleshoot programmatic Kotaemon RAG pipelines with core components, documents, indexes, retrieval, reranking, QA, reasoning, and citations.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- rag-core
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Build and troubleshoot programmatic Kotaemon RAG pipelines with core components, documents, indexes, retrieval, reranking, QA, reasoning, and citations.
- disable-model-invocation
- true
02 资源文件 4
- 参考文档references/api-reference.md13.6 KB
- 参考文档references/rag-workflows.md10.6 KB
- 参考文档references/troubleshooting.md9.7 KB
- 脚本scripts/inspect_pipeline_components.py14.8 KB
03 SKILL.md 正文
Kotaemon RAG Core
Use this sub-skill when an agent needs to compose or debug Kotaemon's programmatic RAG layer: component graphs, document schemas, vector indexing/retrieval, reranking, evidence formatting, prompt templates, reasoning chains, QA output metadata, and citation traces.
Route elsewhere when the task is primarily:
- App setup,
.env,flowsettings.py, Gradio login, Docker, PDF.js, or migration operations: useapp-deployment. - Model credentials, provider endpoints, local LLM servers, embedding model setup, reranker provider setup, or GraphRAG provider toggles: use
model-providers. - Loading/parsing/splitting source files, OCR, table extraction, or parser optional dependencies: use
document-ingestion. - ktem app extensions, custom pages, flow registration, templates, or plugin UI work: use
extensions.
Start Here
- Identify the layer: schema/component contract, indexing, retrieval, reranking, QA/citation, prompt/reasoning, or storage abstraction.
- Use
references/api-reference.mdfor class contracts and import paths. - Use
references/rag-workflows.mdfor composition patterns and metadata-preserving examples. - Use
references/troubleshooting.mdfor common failure modes before changing provider or ingestion code. - Run
scripts/inspect_pipeline_components.py --repo-root <repo>when source or environment drift is suspected.
Core Mental Model
BaseComponentis the composable unit. Implementrun(...); call components directly withcomponent(...)for normal execution.invoke,stream, and async variants exist for components that implement them, butrunis the common subclass contract.Documentcarriescontent,text,metadata, optionalsource, and optionalchannel.RetrievedDocumentextends it withscoreandretrieval_metadata.VectorIndexingembeds documents, writes vectors to aBaseVectorStore, and optionally stores fullDocumentobjects in aBaseDocumentStore.VectorRetrievalembeds the query, fetches ids/scores from the vector store, loads documents from the docstore, returnsRetrievedDocumentobjects, and can apply rerankers.- QA flows convert retrieved documents to evidence text, generate answers, and attach metadata such as
citation,qa_score,mindmap, andcitation_viz.
Difficult Cases
- Low-confidence cited answer: trace
RetrievedDocument.score,retrieval_metadata, reranker output,qa_score, and citation spans throughreferences/rag-workflows.md; route provider scoring failures to../model-providers/SKILL.mdand parser evidence gaps to../document-ingestion/SKILL.md. - Custom component pipeline loses metadata: check the
BaseComponent.run(...)boundary, preserveDocument.metadatawhen transforming chunks, and usescripts/inspect_pipeline_components.pyto confirm the expected schema/index classes are present.
Bundled References
references/api-reference.md- core classes, import paths, method contracts, and metadata expectations.references/rag-workflows.md- indexing/retrieval/QA wiring patterns, prompt and reasoning usage, and safe introspection.references/troubleshooting.md- provider object, embedding/vectorstore mismatch, empty retrieval, metadata loss, citation confidence, prompt variables, missing reranker, and call-style failures.
Bundled Script
python skills/kotaemon/sub-skills/rag-core/scripts/inspect_pipeline_components.py --repo-root <repo-root>
The inspector attempts import-light discovery for key Kotaemon modules, then falls back to AST parsing if imports fail because optional provider dependencies are absent. It reports public classes/functions, base classes, method names, signatures when available, and the discovery mode used per module.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).