memory-rag-and-data
Build CAMEL memory, retrieval/RAG, embedding, storage, loader, datahub, and dataset workflows while keeping data formats, optional extras, and backend constraints explicit.
- License 许可
- Apache-2.0
- 最近更新
- 2026-10-08
- Stars
- ★ 11
- Forks
- 3
01 前言 Front Matter
- name
- memory-rag-and-data
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Build CAMEL memory, retrieval/RAG, embedding, storage, loader, datahub, and dataset workflows while keeping data formats, optional extras, and backend constraints explicit.
- disable-model-invocation
- true
02 资源文件 5
- 参考文档references/api-reference.md9.3 KB
- 参考文档references/data-workflows.md7.1 KB
- 参考文档references/storage-selection.md7.0 KB
- 参考文档references/troubleshooting.md7.5 KB
- 参考文档references/workflows.md7.8 KB
03 SKILL.md 正文
Memory, RAG, And Data
Use this sub-skill when the task is to add memory to CAMEL agents, build a retrieval or RAG pipeline, select embedding/vector/key-value/object/graph storage, ingest files or web data, or validate dataset/datahub flows.
Route By Task
- Agent memory: Read references/workflows.md for
ChatHistoryMemory,VectorDBMemory,LongtermAgentMemory,MemoryRecord,ScoreBasedContextCreator, and attaching memory toChatAgent. - Retrieval/RAG: Read references/workflows.md and references/api-reference.md for
VectorRetriever,AutoRetriever,HybridRetriever, BM25/rerank retrievers, chunking, result schemas, and empty-result debugging. - Storage choice: Read references/storage-selection.md before choosing Qdrant, Chroma, FAISS, Milvus, TiDB, PgVector, Weaviate, Surreal, OceanBase, JSON, Redis, object storage, or graph storage.
- Loaders/data: Read references/data-workflows.md for
BaseIO,UnstructuredIO,MarkItDownLoader, API-backed loaders, Hugging Face datahubs, and dataset generators. - Safe API inspection: Run scripts/inspect_rag_components.py to confirm installed imports, signatures, optional dependencies, and no-network component availability.
- Failures: Read references/troubleshooting.md for extras, Python-version caveats, embedding credentials, vector dimensions, persistence, loader/network keys, graph auth, token budget, and retrieval-empty issues.
Boundary Notes
- Keep provider/model setup minimal here; use the sibling models sub-skill for
ModelFactory,BaseModelBackend, and provider credentials beyond embedding-specific keys. - Keep tool execution and MCP patterns in the tools-oriented sibling sub-skill; this sub-skill only covers data ingestion and retrieval surfaces.
- Keep datagen benchmark/evaluation pipelines in the sibling
datagen-evaluation-and-benchmarkssub-skill; this sub-skill only coverscamel.datasetsand data formats needed by memory/RAG work.
Quick Start Checklist
- Pick an embedding first and record
embedding.get_output_dim(). - Create vector storage with the exact same
vector_dim; decide ephemeral, local persistent path, or remote service. - Ingest local fixtures in CI with
VectorRetriever.process(content=...)or explicitVectorRecordobjects; avoid network loaders in tests. - Query with a positive
top_k, low-enoughsimilarity_threshold, and inspect returned payload keys such astext,metadata, andcontent path. - Attach memory to
ChatAgent(memory=...)or assignagent.memory = memory; validatememory.get_context()fits the model token budget. - Run the bundled inspection script before depending on optional extras such as
rag,storage, ordocument_tools.
04 同仓库更多 Skills
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
obsidian-clipper-template-creator
Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.
python-api
Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP/mAP semantics, and source-style import diagnostics.
file-evaluation
Folder-based PASCAL VOC AP/mAP evaluation from ground-truth and detection text files.
object-detection-metrics
Guides legacy Object-Detection-Metrics PASCAL VOC AP/mAP evaluation from detection text folders and direct Python API classes.
nx-workspace-patterns
Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.