[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7C09PQNJSC3M0R2R5EZS45J":3,"skill-same:01M7C09PQNJSC3M0R2R5EZS45J":50},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":27,"commitSha":28,"skillDirectory":29,"forksCount":30,"repositoryHtmlUrl":31,"repositoryHomepageUrl":32,"ownerHtmlUrl":33,"ownerBlogUrl":6,"ownerDescription":34,"updatedAt":35,"install":36,"resources":49},"01M7C09PQNJSC3M0R2R5EZS45J","embedding-strategies",null,"Guide to selecting and optimizing embedding models for vector search applications.","YYC-Cube\u002FYYC3-Knowledge-Base","YYC-Cube","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F235006543?v=4","01M79J2PH4SWD2X0825X6PN748","yyc3","01M79J2PHVK3F6ZBYT8K2SCZCW",1,"MIT","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-Skills-技能包\u002FYYC3-Skills-技能包-antigravity-awesome-skills\u002Fplugins\u002Fantigravity-awesome-skills-claude\u002Fskills\u002Fembedding-strategies\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: embedding-strategies\ndescription: \"Guide to selecting and optimizing embedding models for vector search applications.\"\nrisk: unknown\nsource: community\ndate_added: \"2026-02-27\"\n---\n\n# Embedding Strategies\n\nGuide to selecting and optimizing embedding models for vector search applications.\n\n## Do not use this skill when\n\n- The task is unrelated to embedding strategies\n- You need a different domain or tool outside this scope\n\n## Instructions\n\n- Clarify goals, constraints, and required inputs.\n- Apply relevant best practices and validate outcomes.\n- Provide actionable steps and verification.\n- If detailed examples are required, open `resources\u002Fimplementation-playbook.md`.\n\n## Use this skill when\n\n- Choosing embedding models for RAG\n- Optimizing chunking strategies\n- Fine-tuning embeddings for domains\n- Comparing embedding model performance\n- Reducing embedding dimensions\n- Handling multilingual content\n\n## Core Concepts\n\n### 1. Embedding Model Comparison\n\n| Model | Dimensions | Max Tokens | Best For |\n|-------|------------|------------|----------|\n| **text-embedding-3-large** | 3072 | 8191 | High accuracy |\n| **text-embedding-3-small** | 1536 | 8191 | Cost-effective |\n| **voyage-2** | 1024 | 4000 | Code, legal |\n| **bge-large-en-v1.5** | 1024 | 512 | Open source |\n| **all-MiniLM-L6-v2** | 384 | 256 | Fast, lightweight |\n| **multilingual-e5-large** | 1024 | 512 | Multi-language |\n\n### 2. Embedding Pipeline\n\n```\nDocument → Chunking → Preprocessing → Embedding Model → Vector\n                ↓\n        [Overlap, Size]  [Clean, Normalize]  [API\u002FLocal]\n```\n\n## Templates\n\n### Template 1: OpenAI Embeddings\n\n```python\nfrom openai import OpenAI\nfrom typing import List\nimport numpy as np\n\nclient = OpenAI()\n\ndef get_embeddings(\n    texts: List[str],\n    model: str = \"text-embedding-3-small\",\n    dimensions: int = None\n) -> List[List[float]]:\n    \"\"\"Get embeddings from OpenAI.\"\"\"\n    # Handle batching for large lists\n    batch_size = 100\n    all_embeddings = []\n\n    for i in range(0, len(texts), batch_size):\n        batch = texts[i:i + batch_size]\n\n        kwargs = {\"input\": batch, \"model\": model}\n        if dimensions:\n            kwargs[\"dimensions\"] = dimensions\n\n        response = client.embeddings.create(**kwargs)\n        embeddings = [item.embedding for item in response.data]\n        all_embeddings.extend(embeddings)\n\n    return all_embeddings\n\n\ndef get_embedding(text: str, **kwargs) -> List[float]:\n    \"\"\"Get single embedding.\"\"\"\n    return get_embeddings([text], **kwargs)[0]\n\n\n# Dimension reduction with OpenAI\ndef get_reduced_embedding(text: str, dimensions: int = 512) -> List[float]:\n    \"\"\"Get embedding with reduced dimensions (Matryoshka).\"\"\"\n    return get_embedding(\n        text,\n        model=\"text-embedding-3-small\",\n        dimensions=dimensions\n    )\n```\n\n### Template 2: Local Embeddings with Sentence Transformers\n\n```python\nfrom sentence_transformers import SentenceTransformer\nfrom typing import List, Optional\nimport numpy as np\n\nclass LocalEmbedder:\n    \"\"\"Local embedding with sentence-transformers.\"\"\"\n\n    def __init__(\n        self,\n        model_name: str = \"BAAI\u002Fbge-large-en-v1.5\",\n        device: str = \"cuda\"\n    ):\n        self.model = SentenceTransformer(model_name, device=device)\n\n    def embed(\n        self,\n        texts: List[str],\n        normalize: bool = True,\n        show_progress: bool = False\n    ) -> np.ndarray:\n        \"\"\"Embed texts with optional normalization.\"\"\"\n        embeddings = self.model.encode(\n            texts,\n            normalize_embeddings=normalize,\n            show_progress_bar=show_progress,\n            convert_to_numpy=True\n        )\n        return embeddings\n\n    def embed_query(self, query: str) -> np.ndarray:\n        \"\"\"Embed a query with BGE-style prefix.\"\"\"\n        # BGE models benefit from query prefix\n        if \"bge\" in self.model.get_sentence_embedding_dimension():\n            query = f\"Represent this sentence for searching relevant passages: {query}\"\n        return self.embed([query])[0]\n\n    def embed_documents(self, documents: List[str]) -> np.ndarray:\n        \"\"\"Embed documents for indexing.\"\"\"\n        return self.embed(documents)\n\n\n# E5 model with instructions\nclass E5Embedder:\n    def __init__(self, model_name: str = \"intfloat\u002Fmultilingual-e5-large\"):\n        self.model = SentenceTransformer(model_name)\n\n    def embed_query(self, query: str) -> np.ndarray:\n        return self.model.encode(f\"query: {query}\")\n\n    def embed_document(self, document: str) -> np.ndarray:\n        return self.model.encode(f\"passage: {document}\")\n```\n\n### Template 3: Chunking Strategies\n\n```python\nfrom typing import List, Tuple\nimport re\n\ndef chunk_by_tokens(\n    text: str,\n    chunk_size: int = 512,\n    chunk_overlap: int = 50,\n    tokenizer=None\n) -> List[str]:\n    \"\"\"Chunk text by token count.\"\"\"\n    import tiktoken\n    tokenizer = tokenizer or tiktoken.get_encoding(\"cl100k_base\")\n\n    tokens = tokenizer.encode(text)\n    chunks = []\n\n    start = 0\n    while start \u003C len(tokens):\n        end = start + chunk_size\n        chunk_tokens = tokens[start:end]\n        chunk_text = tokenizer.decode(chunk_tokens)\n        chunks.append(chunk_text)\n        start = end - chunk_overlap\n\n    return chunks\n\n\ndef chunk_by_sentences(\n    text: str,\n    max_chunk_size: int = 1000,\n    min_chunk_size: int = 100\n) -> List[str]:\n    \"\"\"Chunk text by sentences, respecting size limits.\"\"\"\n    import nltk\n    sentences = nltk.sent_tokenize(text)\n\n    chunks = []\n    current_chunk = []\n    current_size = 0\n\n    for sentence in sentences:\n        sentence_size = len(sentence)\n\n        if current_size + sentence_size > max_chunk_size and current_chunk:\n            chunks.append(\" \".join(current_chunk))\n            current_chunk = []\n            current_size = 0\n\n        current_chunk.append(sentence)\n        current_size += sentence_size\n\n    if current_chunk:\n        chunks.append(\" \".join(current_chunk))\n\n    return chunks\n\n\ndef chunk_by_semantic_sections(\n    text: str,\n    headers_pattern: str = r'^#{1,3}\\s+.+$'\n) -> List[Tuple[str, str]]:\n    \"\"\"Chunk markdown by headers, preserving hierarchy.\"\"\"\n    lines = text.split('\\n')\n    chunks = []\n    current_header = \"\"\n    current_content = []\n\n    for line in lines:\n        if re.match(headers_pattern, line, re.MULTILINE):\n            if current_content:\n                chunks.append((current_header, '\\n'.join(current_content)))\n            current_header = line\n            current_content = []\n        else:\n            current_content.append(line)\n\n    if current_content:\n        chunks.append((current_header, '\\n'.join(current_content)))\n\n    return chunks\n\n\ndef recursive_character_splitter(\n    text: str,\n    chunk_size: int = 1000,\n    chunk_overlap: int = 200,\n    separators: List[str] = None\n) -> List[str]:\n    \"\"\"LangChain-style recursive splitter.\"\"\"\n    separators = separators or [\"\\n\\n\", \"\\n\", \". \", \" \", \"\"]\n\n    def split_text(text: str, separators: List[str]) -> List[str]:\n        if not text:\n            return []\n\n        separator = separators[0]\n        remaining_separators = separators[1:]\n\n        if separator == \"\":\n            # Character-level split\n            return [text[i:i+chunk_size] for i in range(0, len(text), chunk_size - chunk_overlap)]\n\n        splits = text.split(separator)\n        chunks = []\n        current_chunk = []\n        current_length = 0\n\n        for split in splits:\n            split_length = len(split) + len(separator)\n\n            if current_length + split_length > chunk_size and current_chunk:\n                chunk_text = separator.join(current_chunk)\n\n                # Recursively split if still too large\n                if len(chunk_text) > chunk_size and remaining_separators:\n                    chunks.extend(split_text(chunk_text, remaining_separators))\n                else:\n                    chunks.append(chunk_text)\n\n                # Start new chunk with overlap\n                overlap_splits = []\n                overlap_length = 0\n                for s in reversed(current_chunk):\n                    if overlap_length + len(s) \u003C= chunk_overlap:\n                        overlap_splits.insert(0, s)\n                        overlap_length += len(s)\n                    else:\n                        break\n                current_chunk = overlap_splits\n                current_length = overlap_length\n\n            current_chunk.append(split)\n            current_length += split_length\n\n        if current_chunk:\n            chunks.append(separator.join(current_chunk))\n\n        return chunks\n\n    return split_text(text, separators)\n```\n\n### Template 4: Domain-Specific Embedding Pipeline\n\n```python\nclass DomainEmbeddingPipeline:\n    \"\"\"Pipeline for domain-specific embeddings.\"\"\"\n\n    def __init__(\n        self,\n        embedding_model: str = \"text-embedding-3-small\",\n        chunk_size: int = 512,\n        chunk_overlap: int = 50,\n        preprocessing_fn=None\n    ):\n        self.embedding_model = embedding_model\n        self.chunk_size = chunk_size\n        self.chunk_overlap = chunk_overlap\n        self.preprocess = preprocessing_fn or self._default_preprocess\n\n    def _default_preprocess(self, text: str) -> str:\n        \"\"\"Default preprocessing.\"\"\"\n        # Remove excessive whitespace\n        text = re.sub(r'\\s+', ' ', text)\n        # Remove special characters\n        text = re.sub(r'[^\\w\\s.,!?-]', '', text)\n        return text.strip()\n\n    async def process_documents(\n        self,\n        documents: List[dict],\n        id_field: str = \"id\",\n        content_field: str = \"content\",\n        metadata_fields: List[str] = None\n    ) -> List[dict]:\n        \"\"\"Process documents for vector storage.\"\"\"\n        processed = []\n\n        for doc in documents:\n            content = doc[content_field]\n            doc_id = doc[id_field]\n\n            # Preprocess\n            cleaned = self.preprocess(content)\n\n            # Chunk\n            chunks = chunk_by_tokens(\n                cleaned,\n                self.chunk_size,\n                self.chunk_overlap\n            )\n\n            # Create embeddings\n            embeddings = get_embeddings(chunks, self.embedding_model)\n\n            # Create records\n            for i, (chunk, embedding) in enumerate(zip(chunks, embeddings)):\n                record = {\n                    \"id\": f\"{doc_id}_chunk_{i}\",\n                    \"document_id\": doc_id,\n                    \"chunk_index\": i,\n                    \"text\": chunk,\n                    \"embedding\": embedding\n                }\n\n                # Add metadata\n                if metadata_fields:\n                    for field in metadata_fields:\n                        if field in doc:\n                            record[field] = doc[field]\n\n                processed.append(record)\n\n        return processed\n\n\n# Code-specific pipeline\nclass CodeEmbeddingPipeline:\n    \"\"\"Specialized pipeline for code embeddings.\"\"\"\n\n    def __init__(self, model: str = \"voyage-code-2\"):\n        self.model = model\n\n    def chunk_code(self, code: str, language: str) -> List[dict]:\n        \"\"\"Chunk code by functions\u002Fclasses.\"\"\"\n        import tree_sitter\n\n        # Parse with tree-sitter\n        # Extract functions, classes, methods\n        # Return chunks with context\n        pass\n\n    def embed_with_context(self, chunk: str, context: str) -> List[float]:\n        \"\"\"Embed code with surrounding context.\"\"\"\n        combined = f\"Context: {context}\\n\\nCode:\\n{chunk}\"\n        return get_embedding(combined, model=self.model)\n```\n\n### Template 5: Embedding Quality Evaluation\n\n```python\nimport numpy as np\nfrom typing import List, Tuple\n\ndef evaluate_retrieval_quality(\n    queries: List[str],\n    relevant_docs: List[List[str]],  # List of relevant doc IDs per query\n    retrieved_docs: List[List[str]],  # List of retrieved doc IDs per query\n    k: int = 10\n) -> dict:\n    \"\"\"Evaluate embedding quality for retrieval.\"\"\"\n\n    def precision_at_k(relevant: set, retrieved: List[str], k: int) -> float:\n        retrieved_k = retrieved[:k]\n        relevant_retrieved = len(set(retrieved_k) & relevant)\n        return relevant_retrieved \u002F k\n\n    def recall_at_k(relevant: set, retrieved: List[str], k: int) -> float:\n        retrieved_k = retrieved[:k]\n        relevant_retrieved = len(set(retrieved_k) & relevant)\n        return relevant_retrieved \u002F len(relevant) if relevant else 0\n\n    def mrr(relevant: set, retrieved: List[str]) -> float:\n        for i, doc in enumerate(retrieved):\n            if doc in relevant:\n                return 1 \u002F (i + 1)\n        return 0\n\n    def ndcg_at_k(relevant: set, retrieved: List[str], k: int) -> float:\n        dcg = sum(\n            1 \u002F np.log2(i + 2) if doc in relevant else 0\n            for i, doc in enumerate(retrieved[:k])\n        )\n        ideal_dcg = sum(1 \u002F np.log2(i + 2) for i in range(min(len(relevant), k)))\n        return dcg \u002F ideal_dcg if ideal_dcg > 0 else 0\n\n    metrics = {\n        f\"precision@{k}\": [],\n        f\"recall@{k}\": [],\n        \"mrr\": [],\n        f\"ndcg@{k}\": []\n    }\n\n    for relevant, retrieved in zip(relevant_docs, retrieved_docs):\n        relevant_set = set(relevant)\n        metrics[f\"precision@{k}\"].append(precision_at_k(relevant_set, retrieved, k))\n        metrics[f\"recall@{k}\"].append(recall_at_k(relevant_set, retrieved, k))\n        metrics[\"mrr\"].append(mrr(relevant_set, retrieved))\n        metrics[f\"ndcg@{k}\"].append(ndcg_at_k(relevant_set, retrieved, k))\n\n    return {name: np.mean(values) for name, values in metrics.items()}\n\n\ndef compute_embedding_similarity(\n    embeddings1: np.ndarray,\n    embeddings2: np.ndarray,\n    metric: str = \"cosine\"\n) -> np.ndarray:\n    \"\"\"Compute similarity matrix between embedding sets.\"\"\"\n    if metric == \"cosine\":\n        # Normalize\n        norm1 = embeddings1 \u002F np.linalg.norm(embeddings1, axis=1, keepdims=True)\n        norm2 = embeddings2 \u002F np.linalg.norm(embeddings2, axis=1, keepdims=True)\n        return norm1 @ norm2.T\n    elif metric == \"euclidean\":\n        from scipy.spatial.distance import cdist\n        return -cdist(embeddings1, embeddings2, metric='euclidean')\n    elif metric == \"dot\":\n        return embeddings1 @ embeddings2.T\n```\n\n## Best Practices\n\n### Do's\n- **Match model to use case** - Code vs prose vs multilingual\n- **Chunk thoughtfully** - Preserve semantic boundaries\n- **Normalize embeddings** - For cosine similarity\n- **Batch requests** - More efficient than one-by-one\n- **Cache embeddings** - Avoid recomputing\n\n### Don'ts\n- **Don't ignore token limits** - Truncation loses info\n- **Don't mix embedding models** - Incompatible spaces\n- **Don't skip preprocessing** - Garbage in, garbage out\n- **Don't over-chunk** - Lose context\n\n## Resources\n\n- [OpenAI Embeddings](https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fguides\u002Fembeddings)\n- [Sentence Transformers](https:\u002F\u002Fwww.sbert.net\u002F)\n- [MTEB Benchmark](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fmteb\u002Fleaderboard)\n\n## Limitations\n- Use this skill only when the task clearly matches the scope described above.\n- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.\n- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.\n","# Embedding Strategies\n\nGuide to selecting and optimizing embedding models for vector search applications.\n\n## Do not use this skill when\n\n- The task is unrelated to embedding strategies\n- You need a different domain or tool outside this scope\n\n## Instructions\n\n- Clarify goals, constraints, and required inputs.\n- Apply relevant best practices and validate outcomes.\n- Provide actionable steps and verification.\n- If detailed examples are required, open `resources\u002Fimplementation-playbook.md`.\n\n## Use this skill when\n\n- Choosing embedding models for RAG\n- Optimizing chunking strategies\n- Fine-tuning embeddings for domains\n- Comparing embedding model performance\n- Reducing embedding dimensions\n- Handling multilingual content\n\n## Core Concepts\n\n### 1. Embedding Model Comparison\n\n| Model | Dimensions | Max Tokens | Best For |\n|-------|------------|------------|----------|\n| **text-embedding-3-large** | 3072 | 8191 | High accuracy |\n| **text-embedding-3-small** | 1536 | 8191 | Cost-effective |\n| **voyage-2** | 1024 | 4000 | Code, legal |\n| **bge-large-en-v1.5** | 1024 | 512 | Open source |\n| **all-MiniLM-L6-v2** | 384 | 256 | Fast, lightweight |\n| **multilingual-e5-large** | 1024 | 512 | Multi-language |\n\n### 2. Embedding Pipeline\n\n```\nDocument → Chunking → Preprocessing → Embedding Model → Vector\n                ↓\n        [Overlap, Size]  [Clean, Normalize]  [API\u002FLocal]\n```\n\n## Templates\n\n### Template 1: OpenAI Embeddings\n\n```python\nfrom openai import OpenAI\nfrom typing import List\nimport numpy as np\n\nclient = OpenAI()\n\ndef get_embeddings(\n    texts: List[str],\n    model: str = \"text-embedding-3-small\",\n    dimensions: int = None\n) -> List[List[float]]:\n    \"\"\"Get embeddings from OpenAI.\"\"\"\n    # Handle batching for large lists\n    batch_size = 100\n    all_embeddings = []\n\n    for i in range(0, len(texts), batch_size):\n        batch = texts[i:i + batch_size]\n\n        kwargs = {\"input\": batch, \"model\": model}\n        if dimensions:\n            kwargs[\"dimensions\"] = dimensions\n\n        response = client.embeddings.create(**kwargs)\n        embeddings = [item.embedding for item in response.data]\n        all_embeddings.extend(embeddings)\n\n    return all_embeddings\n\n\ndef get_embedding(text: str, **kwargs) -> List[float]:\n    \"\"\"Get single embedding.\"\"\"\n    return get_embeddings([text], **kwargs)[0]\n\n\n# Dimension reduction with OpenAI\ndef get_reduced_embedding(text: str, dimensions: int = 512) -> List[float]:\n    \"\"\"Get embedding with reduced dimensions (Matryoshka).\"\"\"\n    return get_embedding(\n        text,\n        model=\"text-embedding-3-small\",\n        dimensions=dimensions\n    )\n```\n\n### Template 2: Local Embeddings with Sentence Transformers\n\n```python\nfrom sentence_transformers import SentenceTransformer\nfrom typing import List, Optional\nimport numpy as np\n\nclass LocalEmbedder:\n    \"\"\"Local embedding with sentence-transformers.\"\"\"\n\n    def __init__(\n        self,\n        model_name: str = \"BAAI\u002Fbge-large-en-v1.5\",\n        device: str = \"cuda\"\n    ):\n        self.model = SentenceTransformer(model_name, device=device)\n\n    def embed(\n        self,\n        texts: List[str],\n        normalize: bool = True,\n        show_progress: bool = False\n    ) -> np.ndarray:\n        \"\"\"Embed texts with optional normalization.\"\"\"\n        embeddings = self.model.encode(\n            texts,\n            normalize_embeddings=normalize,\n            show_progress_bar=show_progress,\n            convert_to_numpy=True\n        )\n        return embeddings\n\n    def embed_query(self, query: str) -> np.ndarray:\n        \"\"\"Embed a query with BGE-style prefix.\"\"\"\n        # BGE models benefit from query prefix\n        if \"bge\" in self.model.get_sentence_embedding_dimension():\n            query = f\"Represent this sentence for searching relevant passages: {query}\"\n        return self.embed([query])[0]\n\n    def embed_documents(self, documents: List[str]) -> np.ndarray:\n        \"\"\"Embed documents for indexing.\"\"\"\n        return self.embed(documents)\n\n\n# E5 model with instructions\nclass E5Embedder:\n    def __init__(self, model_name: str = \"intfloat\u002Fmultilingual-e5-large\"):\n        self.model = SentenceTransformer(model_name)\n\n    def embed_query(self, query: str) -> np.ndarray:\n        return self.model.encode(f\"query: {query}\")\n\n    def embed_document(self, document: str) -> np.ndarray:\n        return self.model.encode(f\"passage: {document}\")\n```\n\n### Template 3: Chunking Strategies\n\n```python\nfrom typing import List, Tuple\nimport re\n\ndef chunk_by_tokens(\n    text: str,\n    chunk_size: int = 512,\n    chunk_overlap: int = 50,\n    tokenizer=None\n) -> List[str]:\n    \"\"\"Chunk text by token count.\"\"\"\n    import tiktoken\n    tokenizer = tokenizer or tiktoken.get_encoding(\"cl100k_base\")\n\n    tokens = tokenizer.encode(text)\n    chunks = []\n\n    start = 0\n    while start \u003C len(tokens):\n        end = start + chunk_size\n        chunk_tokens = tokens[start:end]\n        chunk_text = tokenizer.decode(chunk_tokens)\n        chunks.append(chunk_text)\n        start = end - chunk_overlap\n\n    return chunks\n\n\ndef chunk_by_sentences(\n    text: str,\n    max_chunk_size: int = 1000,\n    min_chunk_size: int = 100\n) -> List[str]:\n    \"\"\"Chunk text by sentences, respecting size limits.\"\"\"\n    import nltk\n    sentences = nltk.sent_tokenize(text)\n\n    chunks = []\n    current_chunk = []\n    current_size = 0\n\n    for sentence in sentences:\n        sentence_size = len(sentence)\n\n        if current_size + sentence_size > max_chunk_size and current_chunk:\n            chunks.append(\" \".join(current_chunk))\n            current_chunk = []\n            current_size = 0\n\n        current_chunk.append(sentence)\n        current_size += sentence_size\n\n    if current_chunk:\n        chunks.append(\" \".join(current_chunk))\n\n    return chunks\n\n\ndef chunk_by_semantic_sections(\n    text: str,\n    headers_pattern: str = r'^#{1,3}\\s+.+$'\n) -> List[Tuple[str, str]]:\n    \"\"\"Chunk markdown by headers, preserving hierarchy.\"\"\"\n    lines = text.split('\\n')\n    chunks = []\n    current_header = \"\"\n    current_content = []\n\n    for line in lines:\n        if re.match(headers_pattern, line, re.MULTILINE):\n            if current_content:\n                chunks.append((current_header, '\\n'.join(current_content)))\n            current_header = line\n            current_content = []\n        else:\n            current_content.append(line)\n\n    if current_content:\n        chunks.append((current_header, '\\n'.join(current_content)))\n\n    return chunks\n\n\ndef recursive_character_splitter(\n    text: str,\n    chunk_size: int = 1000,\n    chunk_overlap: int = 200,\n    separators: List[str] = None\n) -> List[str]:\n    \"\"\"LangChain-style recursive splitter.\"\"\"\n    separators = separators or [\"\\n\\n\", \"\\n\", \". \", \" \", \"\"]\n\n    def split_text(text: str, separators: List[str]) -> List[str]:\n        if not text:\n            return []\n\n        separator = separators[0]\n        remaining_separators = separators[1:]\n\n        if separator == \"\":\n            # Character-level split\n            return [text[i:i+chunk_size] for i in range(0, len(text), chunk_size - chunk_overlap)]\n\n        splits = text.split(separator)\n        chunks = []\n        current_chunk = []\n        current_length = 0\n\n        for split in splits:\n            split_length = len(split) + len(separator)\n\n            if current_length + split_length > chunk_size and current_chunk:\n                chunk_text = separator.join(current_chunk)\n\n                # Recursively split if still too large\n                if len(chunk_text) > chunk_size and remaining_separators:\n                    chunks.extend(split_text(chunk_text, remaining_separators))\n                else:\n                    chunks.append(chunk_text)\n\n                # Start new chunk with overlap\n                overlap_splits = []\n                overlap_length = 0\n                for s in reversed(current_chunk):\n                    if overlap_length + len(s) \u003C= chunk_overlap:\n                        overlap_splits.insert(0, s)\n                        overlap_length += len(s)\n                    else:\n                        break\n                current_chunk = overlap_splits\n                current_length = overlap_length\n\n            current_chunk.append(split)\n            current_length += split_length\n\n        if current_chunk:\n            chunks.append(separator.join(current_chunk))\n\n        return chunks\n\n    return split_text(text, separators)\n```\n\n### Template 4: Domain-Specific Embedding Pipeline\n\n```python\nclass DomainEmbeddingPipeline:\n    \"\"\"Pipeline for domain-specific embeddings.\"\"\"\n\n    def __init__(\n        self,\n        embedding_model: str = \"text-embedding-3-small\",\n        chunk_size: int = 512,\n        chunk_overlap: int = 50,\n        preprocessing_fn=None\n    ):\n        self.embedding_model = embedding_model\n        self.chunk_size = chunk_size\n        self.chunk_overlap = chunk_overlap\n        self.preprocess = preprocessing_fn or self._default_preprocess\n\n    def _default_preprocess(self, text: str) -> str:\n        \"\"\"Default preprocessing.\"\"\"\n        # Remove excessive whitespace\n        text = re.sub(r'\\s+', ' ', text)\n        # Remove special characters\n        text = re.sub(r'[^\\w\\s.,!?-]', '', text)\n        return text.strip()\n\n    async def process_documents(\n        self,\n        documents: List[dict],\n        id_field: str = \"id\",\n        content_field: str = \"content\",\n        metadata_fields: List[str] = None\n    ) -> List[dict]:\n        \"\"\"Process documents for vector storage.\"\"\"\n        processed = []\n\n        for doc in documents:\n            content = doc[content_field]\n            doc_id = doc[id_field]\n\n            # Preprocess\n            cleaned = self.preprocess(content)\n\n            # Chunk\n            chunks = chunk_by_tokens(\n                cleaned,\n                self.chunk_size,\n                self.chunk_overlap\n            )\n\n            # Create embeddings\n            embeddings = get_embeddings(chunks, self.embedding_model)\n\n            # Create records\n            for i, (chunk, embedding) in enumerate(zip(chunks, embeddings)):\n                record = {\n                    \"id\": f\"{doc_id}_chunk_{i}\",\n                    \"document_id\": doc_id,\n                    \"chunk_index\": i,\n                    \"text\": chunk,\n                    \"embedding\": embedding\n                }\n\n                # Add metadata\n                if metadata_fields:\n                    for field in metadata_fields:\n                        if field in doc:\n                            record[field] = doc[field]\n\n                processed.append(record)\n\n        return processed\n\n\n# Code-specific pipeline\nclass CodeEmbeddingPipeline:\n    \"\"\"Specialized pipeline for code embeddings.\"\"\"\n\n    def __init__(self, model: str = \"voyage-code-2\"):\n        self.model = model\n\n    def chunk_code(self, code: str, language: str) -> List[dict]:\n        \"\"\"Chunk code by functions\u002Fclasses.\"\"\"\n        import tree_sitter\n\n        # Parse with tree-sitter\n        # Extract functions, classes, methods\n        # Return chunks with context\n        pass\n\n    def embed_with_context(self, chunk: str, context: str) -> List[float]:\n        \"\"\"Embed code with surrounding context.\"\"\"\n        combined = f\"Context: {context}\\n\\nCode:\\n{chunk}\"\n        return get_embedding(combined, model=self.model)\n```\n\n### Template 5: Embedding Quality Evaluation\n\n```python\nimport numpy as np\nfrom typing import List, Tuple\n\ndef evaluate_retrieval_quality(\n    queries: List[str],\n    relevant_docs: List[List[str]],  # List of relevant doc IDs per query\n    retrieved_docs: List[List[str]],  # List of retrieved doc IDs per query\n    k: int = 10\n) -> dict:\n    \"\"\"Evaluate embedding quality for retrieval.\"\"\"\n\n    def precision_at_k(relevant: set, retrieved: List[str], k: int) -> float:\n        retrieved_k = retrieved[:k]\n        relevant_retrieved = len(set(retrieved_k) & relevant)\n        return relevant_retrieved \u002F k\n\n    def recall_at_k(relevant: set, retrieved: List[str], k: int) -> float:\n        retrieved_k = retrieved[:k]\n        relevant_retrieved = len(set(retrieved_k) & relevant)\n        return relevant_retrieved \u002F len(relevant) if relevant else 0\n\n    def mrr(relevant: set, retrieved: List[str]) -> float:\n        for i, doc in enumerate(retrieved):\n            if doc in relevant:\n                return 1 \u002F (i + 1)\n        return 0\n\n    def ndcg_at_k(relevant: set, retrieved: List[str], k: int) -> float:\n        dcg = sum(\n            1 \u002F np.log2(i + 2) if doc in relevant else 0\n            for i, doc in enumerate(retrieved[:k])\n        )\n        ideal_dcg = sum(1 \u002F np.log2(i + 2) for i in range(min(len(relevant), k)))\n        return dcg \u002F ideal_dcg if ideal_dcg > 0 else 0\n\n    metrics = {\n        f\"precision@{k}\": [],\n        f\"recall@{k}\": [],\n        \"mrr\": [],\n        f\"ndcg@{k}\": []\n    }\n\n    for relevant, retrieved in zip(relevant_docs, retrieved_docs):\n        relevant_set = set(relevant)\n        metrics[f\"precision@{k}\"].append(precision_at_k(relevant_set, retrieved, k))\n        metrics[f\"recall@{k}\"].append(recall_at_k(relevant_set, retrieved, k))\n        metrics[\"mrr\"].append(mrr(relevant_set, retrieved))\n        metrics[f\"ndcg@{k}\"].append(ndcg_at_k(relevant_set, retrieved, k))\n\n    return {name: np.mean(values) for name, values in metrics.items()}\n\n\ndef compute_embedding_similarity(\n    embeddings1: np.ndarray,\n    embeddings2: np.ndarray,\n    metric: str = \"cosine\"\n) -> np.ndarray:\n    \"\"\"Compute similarity matrix between embedding sets.\"\"\"\n    if metric == \"cosine\":\n        # Normalize\n        norm1 = embeddings1 \u002F np.linalg.norm(embeddings1, axis=1, keepdims=True)\n        norm2 = embeddings2 \u002F np.linalg.norm(embeddings2, axis=1, keepdims=True)\n        return norm1 @ norm2.T\n    elif metric == \"euclidean\":\n        from scipy.spatial.distance import cdist\n        return -cdist(embeddings1, embeddings2, metric='euclidean')\n    elif metric == \"dot\":\n        return embeddings1 @ embeddings2.T\n```\n\n## Best Practices\n\n### Do's\n- **Match model to use case** - Code vs prose vs multilingual\n- **Chunk thoughtfully** - Preserve semantic boundaries\n- **Normalize embeddings** - For cosine similarity\n- **Batch requests** - More efficient than one-by-one\n- **Cache embeddings** - Avoid recomputing\n\n### Don'ts\n- **Don't ignore token limits** - Truncation loses info\n- **Don't mix embedding models** - Incompatible spaces\n- **Don't skip preprocessing** - Garbage in, garbage out\n- **Don't over-chunk** - Lose context\n\n## Resources\n\n- [OpenAI Embeddings](https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fguides\u002Fembeddings)\n- [Sentence Transformers](https:\u002F\u002Fwww.sbert.net\u002F)\n- [MTEB Benchmark](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fmteb\u002Fleaderboard)\n\n## Limitations\n- Use this skill only when the task clearly matches the scope described above.\n- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.\n- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.\n",{"name":5,"risk":24,"source":25,"date_added":26,"description":7},"unknown","community","2026-02-27","main","87afe6ce96220accf50a345dea46784c8c9f210e","YYC3-Skills-技能包\u002FYYC3-Skills-技能包-antigravity-awesome-skills\u002Fplugins\u002Fantigravity-awesome-skills-claude\u002Fskills\u002Fembedding-strategies",0,"https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base","https:\u002F\u002Fkb.yyc3.top\u002F","https:\u002F\u002Fgithub.com\u002FYYC-Cube","万象归元于云枢 | 深栈智启新纪元","2026-11-10 10:37:32",{"source":8,"skillName":5,"commands":37,"prompt":47,"zipAvailable":48},[38,41,44],{"tool":39,"command":40},"npx","npx skills add YYC-Cube\u002FYYC3-Knowledge-Base --skill embedding-strategies",{"tool":42,"command":43},"bunx","bunx skills add YYC-Cube\u002FYYC3-Knowledge-Base --skill embedding-strategies",{"tool":45,"command":46},"pnpm","pnpm dlx skills add YYC-Cube\u002FYYC3-Knowledge-Base --skill embedding-strategies","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-Skills-技能包\u002FYYC3-Skills-技能包-antigravity-awesome-skills\u002Fplugins\u002Fantigravity-awesome-skills-claude\u002Fskills\u002Fembedding-strategies\u002FSKILL.md",true,[],[51,58,65,72,79,86],{"skillNo":52,"name":53,"nameZh":6,"description":54,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":55,"categories":56,"keywords":57,"publishedAt":19},"01M7CTDWHERCXPQCP0K03A86AS","vercel-sandbox","Run agent-browser + Chrome inside Vercel Sandbox microVMs for browser automation from any Vercel-deployed app. Use when the user needs browser automation in a Vercel app (Next.js, SvelteKit, Nuxt, Remix, Astro, etc.), wants to run headless Chrome without binary size limits, needs persistent browser sessions across commands, or wants ephemeral isolated browser environments. Triggers include \"Vercel Sandbox browser\", \"microVM Chrome\", \"agent-browser in sandbox\", \"browser automation on Vercel\", or any task requiring Chrome in a Vercel Sandbox.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Fvercel-sandbox\u002FSKILL.md",[],[],{"skillNo":59,"name":60,"nameZh":6,"description":61,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":62,"categories":63,"keywords":64,"publishedAt":19},"01M7CTD8D2BNJY54S65M47HEJA","slack","Interact with Slack workspaces using browser automation. Use when the user needs to check unread channels, navigate Slack, send messages, extract data, find information, search conversations, or automate any Slack task. Triggers include \"check my Slack\", \"what channels have unreads\", \"send a message to\", \"search Slack for\", \"extract from Slack\", \"find who said\", or any task requiring programmatic Slack interaction.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Fslack\u002FSKILL.md",[],[],{"skillNo":66,"name":67,"nameZh":6,"description":68,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":69,"categories":70,"keywords":71,"publishedAt":19},"01M7CTCSSB8ZP3Q58C3YZDZK1W","electron","Automate Electron desktop apps (VS Code, Slack, Discord, Figma, Notion, Spotify, etc.) using agent-browser via Chrome DevTools Protocol. Use when the user needs to interact with an Electron app, automate a desktop app, connect to a running app, control a native app, or test an Electron application. Triggers include \"automate Slack app\", \"control VS Code\", \"interact with Discord app\", \"test this Electron app\", \"connect to desktop app\", or any task requiring automation of a native Electron application.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Felectron\u002FSKILL.md",[],[],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":19},"01M7CTCBJY76688V4DCZGWFHGQ","dogfood","Systematically explore and test a web application to find bugs, UX issues, and other problems. Use when asked to \"dogfood\", \"QA\", \"exploratory test\", \"find issues\", \"bug hunt\", \"test this app\u002Fsite\u002Fplatform\", or review the quality of a web application. Produces a structured report with full reproduction evidence -- step-by-step screenshots, repro videos, and detailed repro steps for every issue -- so findings can be handed directly to the responsible teams.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Fdogfood\u002FSKILL.md",[],[],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":19},"01M7CTC4H0FFGVEAQR9PA4GA1A","agent-browser","Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to \"open a website\", \"fill out a form\", \"click a button\", \"take a screenshot\", \"scrape data from a page\", \"test this web app\", \"login to a site\", \"automate browser actions\", or any task requiring programmatic web interaction.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-WebSpec-Web标准\u002FYYC3-WebSpec-Web标准-standards\u002Fbrowser-agent\u002Fskills\u002Fagent-browser\u002FSKILL.md",[],[],{"skillNo":87,"name":88,"nameZh":6,"description":89,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":19},"01M7CTBKVFYBG78FDSABSDJTTQ","shadcn","Manages shadcn components and projects — adding, searching, fixing, debugging, styling, and composing UI, including chat interfaces. Provides project context, component docs, and usage examples. Applies when working with shadcn\u002Fui, component registries, presets, --preset codes, or any project with a components.json file. Also triggers for \"shadcn init\", \"create an app with --preset\", \"switch to --preset\", and for building a design system or applying a DESIGN.md or brand spec to shadcn\u002Fui.","https:\u002F\u002Fgithub.com\u002FYYC-Cube\u002FYYC3-Knowledge-Base\u002Fblob\u002Fmain\u002FYYC3-UI-界面组件\u002FYYC3-UI-界面组件-ui\u002Fskills\u002Fshadcn\u002FSKILL.md",[],[]]