[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6GGBSXFSZE1504EMMZKC87R":3,"skill-same:01M6GGBSXFSZE1504EMMZKC87R":69},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":49,"repositoryHtmlUrl":50,"repositoryHomepageUrl":6,"ownerHtmlUrl":51,"ownerBlogUrl":52,"ownerDescription":53,"updatedAt":54,"install":55,"resources":68},"01M6GGBSXFSZE1504EMMZKC87R","rag",null,"Build multi-modal RAG pipelines with graph-enhanced retrieval in Go using saige\u002Frag. Covers chunking, multi-retriever fusion, reranking, HyDE, evaluation metrics, and PostgreSQL + pgvector storage.","urmzd\u002Fsaige","urmzd","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F45431570?v=4","01M6GGAZNB0MWCE67F32JA7FKS","Urmzd Mukhammadnaim","01M6GGAZSYQ5Z87X42HF8CCWBM",6,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige\u002Fblob\u002Fmain\u002Fskills\u002Frag\u002FSKILL.md",[],[5,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37],"knowledge","with","pgvector","multi","retrieval","HyDE","fusion","chunking","build","pipelines","go","knowledge-graph","modal","Evaluation","graph","embeddings","metrics","postgresql","storage","2026-11-10 10:12:30",false,"---\nname: rag\ndescription: Build multi-modal RAG pipelines with graph-enhanced retrieval in Go using saige\u002Frag. Covers chunking, multi-retriever fusion, reranking, HyDE, evaluation metrics, and PostgreSQL + pgvector storage.\nmetadata:\n  version: 0.1.0\n  author: urmzd\n  tags: rag retrieval embeddings knowledge-graph go saige\n---\n\n# rag\n\nA Go library for multi-modal Retrieval-Augmented Generation with graph-enhanced retrieval, part of the saige SDK.\n\n## What it does\n\nsaige\u002Frag models documents as hierarchical structures (Document -> Section -> ContentVariant) where each section can have multiple modality representations (text, image, table, audio). It provides a pluggable pipeline for ingesting documents, generating embeddings, performing hybrid search (vector + BM25 + graph), reranking, and assembling context with citations.\n\n## When to use\n\n- Ingesting documents (PDF, text, images) into a searchable vector store\n- Building RAG pipelines with multi-modal content support\n- Combining vector search with knowledge graph traversal\n- Evaluating retrieval and generation quality\n- Deduplicating documents by content fingerprint\n\n## Usage\n\n### Install\n\n```bash\ngo get github.com\u002Furmzd\u002Fsaige\n```\n\n### Create a pipeline\n\n```go\nimport (\n    \"github.com\u002Furmzd\u002Fsaige\u002Frag\"\n    \"github.com\u002Furmzd\u002Fsaige\u002Frag\u002Fpgstore\"\n    \"github.com\u002Furmzd\u002Fsaige\u002Frag\u002Ftypes\"\n    \"github.com\u002Furmzd\u002Fsaige\u002Fpostgres\"\n)\n\n\u002F\u002F Connect to PostgreSQL 18 (requires the pgvector and pg_search extensions).\npool, _ := postgres.NewPool(ctx, postgres.Config{URL: \"postgres:\u002F\u002Flocalhost:5432\u002Fmydb\"})\npostgres.RunMigrations(ctx, pool, postgres.MigrationOptions{})\n\nstore, _ := pgstore.New(pgstore.Config{Pool: pool})\npipe, err := rag.New(rag.Config{Store: store},\n    rag.WithContentExtractor(myExtractor),\n    rag.WithEmbedders(myEmbedderRegistry),\n    rag.WithRecursiveChunker(512, 50),\n    rag.WithBM25(nil),\n    rag.WithMMR(0.7),\n)\n```\n\n### Ingest a document\n\n```go\nresult, err := pipe.Ingest(ctx, &types.RawDocument{\n    SourceURI: \"https:\u002F\u002Fexample.com\u002Fpaper.pdf\",\n    MIMEType:  \"application\u002Fpdf\",\n    Data:      pdfBytes,\n})\n```\n\n### Search\n\n```go\nresults, err := pipe.Search(ctx, \"attention mechanism\", types.WithLimit(5), types.WithContextAssembly(4096))\nfmt.Println(results.Context.Prompt) \u002F\u002F context with citations\n```\n\n### With knowledge graph integration\n\n```go\npipe, err := rag.New(rag.Config{Store: store},\n    rag.WithContentExtractor(extractor),\n    rag.WithGraph(kgGraph),  \u002F\u002F enables entity extraction + graph retrieval\n)\n```\n\n## Key interfaces\n\n| Interface | Purpose |\n|-----------|---------|\n| `Pipeline` | Orchestrate ingest, search, update, delete, reconstruct |\n| `Store` | Document CRUD + vector search |\n| `ContentExtractor` | Raw bytes -> structured Document |\n| `Chunker` | Split long sections |\n| `EmbedderRegistry` | Dispatch embedding by ContentType |\n\n## Configuration options\n\n| Option | Purpose |\n|--------|---------|\n| `WithStore(s)` | Set the document store, or set `rag.Config.Store` (required) |\n| `WithContentExtractor(ext)` | Set the content extractor (required) |\n| `WithRecursiveChunker(max, overlap)` | Recursive text chunking |\n| `WithSemanticChunker(thresh, min, max)` | Semantic similarity chunking |\n| `WithEmbedders(reg)` | Set the embedder registry (optional, needed for search) |\n| `WithGraph(g)` | Enable knowledge graph entity extraction (optional) |\n| `WithBM25(cfg)` | Enable BM25 lexical retrieval. Over pgstore it searches the store's pg_search index; over a store without keyword search the index is in memory, so call `rag.RebuildIndex(ctx, pipe)` once after `rag.New` when the store already holds documents |\n| `WithParentContext()` | Expand hits to parent section context |\n| `WithMMR(lambda)` | MMR diversity reranking |\n| `WithCrossEncoder(scorer)` | Cross-encoder reranking |\n| `WithHyDE(llm, n)` | HyDE query expansion |\n| `WithCompression(llm)` | LLM-based context compression |\n| `WithDedupBehavior(b)` | Set dedup behavior: DedupSkip (default) or DedupReplace |\n| `WithStoreOriginals(true)` | Persist raw document bytes |\n\nSearch options: `types.WithCandidatePool(k)` sets how many fused candidates reach the reranker, and `types.WithMinScore(s)` is a threshold in [0, 1] on the normalized fused score (1.0 keeps only hits ranked first by every retriever), applied before reranking.\n\n## Evaluation\n\n```go\nimport \"github.com\u002Furmzd\u002Fsaige\u002Frag\u002Feval\"\n\n\u002F\u002F Retrieval metrics (pure functions).\nprecision := eval.ContextPrecision(hits, relevantUUIDs)\nrecall := eval.ContextRecall(hits, relevantUUIDs)\nndcg := eval.NDCG(hits, relevantUUIDs, 10)\nmrr := eval.MRR(hits, relevantUUIDs)\n\n\u002F\u002F Generation metrics (require LLM).\nfaith, _, _ := eval.Faithfulness(ctx, response, contextText, llm)\ncorrectness, _ := eval.AnswerCorrectness(ctx, response, groundTruth, llm)\n```\n\n## Agent tool bindings\n\n```go\nimport \"github.com\u002Furmzd\u002Fsaige\u002Frag\u002Ftool\"\n\ntools := tool.NewTools(pipeline)\n\u002F\u002F rag_search, rag_lookup, rag_update, rag_delete, rag_reconstruct\n```\n\n## CLI\n\n```bash\nsaige knowledge ingest --db \"$SAIGE_RAG_DB\" --file paper.pdf --mime application\u002Fpdf\nsaige knowledge sync --db \"$SAIGE_RAG_DB\" --dir docs --include '**\u002F*.md' --exclude 'drafts\u002F**' --prune\nsaige knowledge search --db \"$SAIGE_RAG_DB\" --query \"attention mechanism\"\nsaige knowledge show --db \"$SAIGE_RAG_DB\" --uuid \u003Cvariant-uuid>\nsaige knowledge delete --db \"$SAIGE_RAG_DB\" --uuid \u003Cdoc-uuid>\n```\n","# rag\n\nA Go library for multi-modal Retrieval-Augmented Generation with graph-enhanced retrieval, part of the saige SDK.\n\n## What it does\n\nsaige\u002Frag models documents as hierarchical structures (Document -> Section -> ContentVariant) where each section can have multiple modality representations (text, image, table, audio). It provides a pluggable pipeline for ingesting documents, generating embeddings, performing hybrid search (vector + BM25 + graph), reranking, and assembling context with citations.\n\n## When to use\n\n- Ingesting documents (PDF, text, images) into a searchable vector store\n- Building RAG pipelines with multi-modal content support\n- Combining vector search with knowledge graph traversal\n- Evaluating retrieval and generation quality\n- Deduplicating documents by content fingerprint\n\n## Usage\n\n### Install\n\n```bash\ngo get github.com\u002Furmzd\u002Fsaige\n```\n\n### Create a pipeline\n\n```go\nimport (\n    \"github.com\u002Furmzd\u002Fsaige\u002Frag\"\n    \"github.com\u002Furmzd\u002Fsaige\u002Frag\u002Fpgstore\"\n    \"github.com\u002Furmzd\u002Fsaige\u002Frag\u002Ftypes\"\n    \"github.com\u002Furmzd\u002Fsaige\u002Fpostgres\"\n)\n\n\u002F\u002F Connect to PostgreSQL 18 (requires the pgvector and pg_search extensions).\npool, _ := postgres.NewPool(ctx, postgres.Config{URL: \"postgres:\u002F\u002Flocalhost:5432\u002Fmydb\"})\npostgres.RunMigrations(ctx, pool, postgres.MigrationOptions{})\n\nstore, _ := pgstore.New(pgstore.Config{Pool: pool})\npipe, err := rag.New(rag.Config{Store: store},\n    rag.WithContentExtractor(myExtractor),\n    rag.WithEmbedders(myEmbedderRegistry),\n    rag.WithRecursiveChunker(512, 50),\n    rag.WithBM25(nil),\n    rag.WithMMR(0.7),\n)\n```\n\n### Ingest a document\n\n```go\nresult, err := pipe.Ingest(ctx, &types.RawDocument{\n    SourceURI: \"https:\u002F\u002Fexample.com\u002Fpaper.pdf\",\n    MIMEType:  \"application\u002Fpdf\",\n    Data:      pdfBytes,\n})\n```\n\n### Search\n\n```go\nresults, err := pipe.Search(ctx, \"attention mechanism\", types.WithLimit(5), types.WithContextAssembly(4096))\nfmt.Println(results.Context.Prompt) \u002F\u002F context with citations\n```\n\n### With knowledge graph integration\n\n```go\npipe, err := rag.New(rag.Config{Store: store},\n    rag.WithContentExtractor(extractor),\n    rag.WithGraph(kgGraph),  \u002F\u002F enables entity extraction + graph retrieval\n)\n```\n\n## Key interfaces\n\n| Interface | Purpose |\n|-----------|---------|\n| `Pipeline` | Orchestrate ingest, search, update, delete, reconstruct |\n| `Store` | Document CRUD + vector search |\n| `ContentExtractor` | Raw bytes -> structured Document |\n| `Chunker` | Split long sections |\n| `EmbedderRegistry` | Dispatch embedding by ContentType |\n\n## Configuration options\n\n| Option | Purpose |\n|--------|---------|\n| `WithStore(s)` | Set the document store, or set `rag.Config.Store` (required) |\n| `WithContentExtractor(ext)` | Set the content extractor (required) |\n| `WithRecursiveChunker(max, overlap)` | Recursive text chunking |\n| `WithSemanticChunker(thresh, min, max)` | Semantic similarity chunking |\n| `WithEmbedders(reg)` | Set the embedder registry (optional, needed for search) |\n| `WithGraph(g)` | Enable knowledge graph entity extraction (optional) |\n| `WithBM25(cfg)` | Enable BM25 lexical retrieval. Over pgstore it searches the store's pg_search index; over a store without keyword search the index is in memory, so call `rag.RebuildIndex(ctx, pipe)` once after `rag.New` when the store already holds documents |\n| `WithParentContext()` | Expand hits to parent section context |\n| `WithMMR(lambda)` | MMR diversity reranking |\n| `WithCrossEncoder(scorer)` | Cross-encoder reranking |\n| `WithHyDE(llm, n)` | HyDE query expansion |\n| `WithCompression(llm)` | LLM-based context compression |\n| `WithDedupBehavior(b)` | Set dedup behavior: DedupSkip (default) or DedupReplace |\n| `WithStoreOriginals(true)` | Persist raw document bytes |\n\nSearch options: `types.WithCandidatePool(k)` sets how many fused candidates reach the reranker, and `types.WithMinScore(s)` is a threshold in [0, 1] on the normalized fused score (1.0 keeps only hits ranked first by every retriever), applied before reranking.\n\n## Evaluation\n\n```go\nimport \"github.com\u002Furmzd\u002Fsaige\u002Frag\u002Feval\"\n\n\u002F\u002F Retrieval metrics (pure functions).\nprecision := eval.ContextPrecision(hits, relevantUUIDs)\nrecall := eval.ContextRecall(hits, relevantUUIDs)\nndcg := eval.NDCG(hits, relevantUUIDs, 10)\nmrr := eval.MRR(hits, relevantUUIDs)\n\n\u002F\u002F Generation metrics (require LLM).\nfaith, _, _ := eval.Faithfulness(ctx, response, contextText, llm)\ncorrectness, _ := eval.AnswerCorrectness(ctx, response, groundTruth, llm)\n```\n\n## Agent tool bindings\n\n```go\nimport \"github.com\u002Furmzd\u002Fsaige\u002Frag\u002Ftool\"\n\ntools := tool.NewTools(pipeline)\n\u002F\u002F rag_search, rag_lookup, rag_update, rag_delete, rag_reconstruct\n```\n\n## CLI\n\n```bash\nsaige knowledge ingest --db \"$SAIGE_RAG_DB\" --file paper.pdf --mime application\u002Fpdf\nsaige knowledge sync --db \"$SAIGE_RAG_DB\" --dir docs --include '**\u002F*.md' --exclude 'drafts\u002F**' --prune\nsaige knowledge search --db \"$SAIGE_RAG_DB\" --query \"attention mechanism\"\nsaige knowledge show --db \"$SAIGE_RAG_DB\" --uuid \u003Cvariant-uuid>\nsaige knowledge delete --db \"$SAIGE_RAG_DB\" --uuid \u003Cdoc-uuid>\n```\n",{"name":5,"metadata":43,"description":7},{"tags":44,"author":9,"version":45},"rag retrieval embeddings knowledge-graph go saige","0.1.0","main","0cbbe64f73683ddba233863b9f3074588e9d73f2","skills\u002Frag",0,"https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige","https:\u002F\u002Fgithub.com\u002Furmzd","https:\u002F\u002Furmzd.com","I like sharing knowledge - usually through the weird things I build here.","2026-11-12 02:37:22",{"source":8,"skillName":5,"commands":56,"prompt":66,"zipAvailable":67},[57,60,63],{"tool":58,"command":59},"npx","npx skills add urmzd\u002Fsaige --skill rag",{"tool":61,"command":62},"bunx","bunx skills add urmzd\u002Fsaige --skill rag",{"tool":64,"command":65},"pnpm","pnpm dlx skills add urmzd\u002Fsaige --skill rag","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige\u002Fblob\u002Fmain\u002Fskills\u002Frag\u002FSKILL.md",true,[],[70,93,120,129,140,153],{"skillNo":71,"name":30,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":80,"publishedAt":38},"01M6GGBP3ZF894HT4FSZQJD0SN","Build and query knowledge graphs with PostgreSQL + pgvector. Ingest episodes, extract entities\u002Frelations via LLM, and search facts by semantic similarity or keyword. Use when working with knowledge graphs, entity extraction, or graph storage.","https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige\u002Fblob\u002Fmain\u002Fskills\u002Fknowledge-graph\u002FSKILL.md",[75],{"categoryNo":76,"categoryName":77,"taxonomyType":78,"taxonomyTypeDesc":79},"01JBSEEDCAT000000000000021","资料检索与研究","SCENARIO","使用场景",[19,81,33,37,36,82,83,20,84,85,86,27,87,88,89,90,91,92,21],"knowledge graph","search","extraction","extract","use","ingest","query","graphs","semantic","similarity","via","Semantic Similarity",{"skillNo":94,"name":95,"nameZh":6,"description":96,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":97,"categories":98,"keywords":102,"publishedAt":38},"01M6GGBJWXFS6XD8FYBZAYY2C5","agent","Build streaming LLM agent loops in Go with typed parts and part deltas, tool execution, media input, context compaction, sub-agent delegation, and RLHF feedback. Use when building AI agents, integrating LLM providers, or implementing tool-use patterns.","https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige\u002Fblob\u002Fmain\u002Fskills\u002Fagent\u002FSKILL.md",[99],{"categoryNo":100,"categoryName":101,"taxonomyType":78,"taxonomyTypeDesc":79},"01JBSEEDCAT000000000000023","Skill与智能体开发",[103,20,104,105,106,107,29,108,109,110,111,112,113,114,115,116,117,85,118,119],"loops","compaction","input","patterns","delegation","ai agents","task","streaming","RLHF","Providers","execution","Media","tool","feedback","context","deltas","llm agent",{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":126,"publishedAt":38},"01M6GGBG7YK3ZHAAGKV42G5VH6","review","Codex copy, shadowed by the Claude one.","https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige\u002Fblob\u002Fmain\u002Fcmd\u002Fsaige\u002Ftestdata\u002Fimport\u002Fproject\u002F.codex\u002Fskills\u002Freview\u002FSKILL.md",[],[122,127,128],"codex","copy",{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":133,"categories":134,"keywords":138,"publishedAt":38},"01M6GGBDJ573185PRVSFZ2H56H","lint","Run the linters.","https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige\u002Fblob\u002Fmain\u002Fcmd\u002Fsaige\u002Ftestdata\u002Fimport\u002Fproject\u002F.codex\u002Fskills\u002Flint\u002FSKILL.md",[135],{"categoryNo":136,"categoryName":137,"taxonomyType":78,"taxonomyTypeDesc":79},"01JBSEEDCAT000000000000007","代码审查",[131,139],"run",{"skillNo":141,"name":122,"nameZh":6,"description":142,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":143,"licenseFileUrl":6,"sourceHtmlUrl":144,"categories":145,"keywords":146,"publishedAt":38},"01M6GGBAN3N6CQZ2AD7DS73HD7","Review a diff for correctness.","MIT","https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige\u002Fblob\u002Fmain\u002Fcmd\u002Fsaige\u002Ftestdata\u002Fimport\u002Fproject\u002F.claude\u002Fskills\u002Freview\u002FSKILL.md",[],[122,147,148,149,150,151,152],"grep","read","diff","platform","correctness","true",{"skillNo":154,"name":155,"nameZh":6,"description":156,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":157,"categories":158,"keywords":160,"publishedAt":38},"01M6GGB7J1MQGVD899KHP3J7HR","hand","Claude copy of a skill already in .agents\u002Fskills.","https:\u002F\u002Fgithub.com\u002Furmzd\u002Fsaige\u002Fblob\u002Fmain\u002Fcmd\u002Fsaige\u002Ftestdata\u002Fimport\u002Fproject\u002F.claude\u002Fskills\u002Fhand\u002FSKILL.md",[159],{"categoryNo":100,"categoryName":101,"taxonomyType":78,"taxonomyTypeDesc":79},[155,128]]