[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6P82PYTPX0J8769GJ6SAXCB":3,"skill-same:01M6P82PYTPX0J8769GJ6SAXCB":76},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":31,"commitSha":57,"skillDirectory":58,"forksCount":13,"repositoryHtmlUrl":59,"repositoryHomepageUrl":6,"ownerHtmlUrl":60,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":61,"install":62,"resources":75},"01M6P82PYTPX0J8769GJ6SAXCB","llm-caching",null,"Implement multi-layer LLM caching with exact match, semantic similarity, and provider-side prompt caching.","duc-duy-1612\u002FOnline-Delivery-Systems-Web","duc-duy-1612","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F192231384?v=4","01M6NSDKQQDKVTHZ8DZMZ1VYY6","01M6NSDKW8DDVTQHBT1HBDDJPT",0,"MIT","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fllm-caching\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"llm caching","caching","provider","layer","relevant","scripts","critical","platform","main","community","security","only","environment","access","devops-security","prompt caching","kubectl","ci","runners","git","2026-11-10 10:12:30",false,"---\nname: llm-caching\ndescription: Implement multi-layer LLM caching with exact match, semantic similarity,\n  and provider-side prompt caching.\ncategory: devops\nrisk: critical\nsource: https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\nsource_repo: BagelHole\u002FDevOps-Security-Agent-Skills\nsource_type: community\ndate_added: '2026-09-20'\nlicense: MIT\nlicense_source: https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\u002Fblob\u002Fmain\u002FLICENSE\ncompatibility: Requires the relevant platform CLIs (kubectl, helm, terraform, git,\n  CI runners) and authorized access to the target environment. Docs-only; helper scripts\n  and templates not bundled.\nmetadata:\n  author: devops-skills\n  version: '1.0'\n---\n\n# LLM Caching\n\nCut LLM costs and latency with exact match, semantic, and provider-side caching layers.\n\n## When to Use This Skill\n\nUse this skill when:\n- The same or similar queries are asked repeatedly (FAQ bots, support tools)\n- LLM API costs are growing and you need immediate savings\n- Serving high request volumes where repeated queries cause bottlenecks\n- Implementing prompt caching for long system prompts (Anthropic\u002FOpenAI)\n- Building offline-capable AI features that need response persistence\n\n## Caching Layers\n\n```\nRequest → Exact Cache → Semantic Cache → Provider Cache → LLM API\n             ↓ hit            ↓ hit             ↓ hit\n           instant          ~5ms           50-80% cheaper\n```\n\n## Layer 1: Exact Match Cache (Redis)\n\n```python\nimport hashlib\nimport json\nimport redis\nfrom openai import OpenAI\n\nr = redis.Redis(host=\"localhost\", port=6379, decode_responses=True)\nclient = OpenAI()\n\ndef build_cache_key(model: str, messages: list, temperature: float) -> str:\n    \"\"\"Deterministic key from request parameters.\"\"\"\n    payload = json.dumps({\n        \"model\": model,\n        \"messages\": messages,\n        \"temperature\": temperature,\n    }, sort_keys=True)\n    return f\"llm:exact:{hashlib.sha256(payload.encode()).hexdigest()}\"\n\ndef cached_completion(model: str, messages: list, temperature: float = 0.0,\n                      ttl: int = 3600) -> dict:\n    key = build_cache_key(model, messages, temperature)\n\n    # Check cache\n    if cached := r.get(key):\n        return json.loads(cached)\n\n    # Call API\n    response = client.chat.completions.create(\n        model=model, messages=messages, temperature=temperature\n    )\n    result = response.model_dump()\n\n    # Cache result (only cache deterministic responses)\n    if temperature == 0.0:\n        r.setex(key, ttl, json.dumps(result))\n\n    return result\n```\n\n## Layer 2: Semantic Cache (GPTCache)\n\n```python\nfrom gptcache import cache, Config\nfrom gptcache.adapter import openai\nfrom gptcache.embedding import Onnx\nfrom gptcache.manager import CacheBase, VectorBase, get_data_manager\nfrom gptcache.similarity_evaluation.distance import SearchDistanceEvaluation\n\n# Configure GPTCache with Qdrant backend\ndef init_gptcache(cache_obj, llm: str):\n    onnx = Onnx()                              # local embedding model\n    data_manager = get_data_manager(\n        CacheBase(\"redis\"),                    # metadata store\n        VectorBase(\"qdrant\",\n                   host=\"localhost\",\n                   port=6333,\n                   collection_name=f\"llm-cache-{llm}\",\n                   dimension=onnx.dimension),\n    )\n    cache_obj.init(\n        embedding_func=onnx.to_embeddings,\n        data_manager=data_manager,\n        similarity_evaluation=SearchDistanceEvaluation(),\n        config=Config(similarity_threshold=0.80),  # 80% similarity = cache hit\n    )\n\ncache.set_openai_key()\ninit_gptcache(cache, \"gpt-4o-mini\")\n\n# Now openai calls are automatically cached\nresponse = openai.ChatCompletion.create(\n    model=\"gpt-4o-mini\",\n    messages=[{\"role\": \"user\", \"content\": \"What is machine learning?\"}],\n)\n# Second call with similar question (\"Explain machine learning\") → cache hit\n```\n\n## Custom Semantic Cache (Production-Grade)\n\n```python\nfrom sentence_transformers import SentenceTransformer\nfrom qdrant_client import QdrantClient\nfrom qdrant_client.models import Distance, VectorParams, PointStruct, Filter, FieldCondition, Range\nimport numpy as np\nimport uuid\nimport time\n\nembed_model = SentenceTransformer(\"BAAI\u002Fbge-small-en-v1.5\")  # fast, 33M params\nqdrant = QdrantClient(\"http:\u002F\u002Flocalhost:6333\")\n\nCACHE_COLLECTION = \"semantic-cache\"\nSIMILARITY_THRESHOLD = 0.88\nCACHE_TTL_SECONDS = 86400  # 24h\n\n# Create collection once\nqdrant.create_collection(\n    collection_name=CACHE_COLLECTION,\n    vectors_config=VectorParams(size=384, distance=Distance.COSINE),\n    on_disk_payload=True,\n)\n\ndef semantic_cache_lookup(query: str, model: str) -> str | None:\n    embedding = embed_model.encode(query).tolist()\n    results = qdrant.query_points(\n        collection_name=CACHE_COLLECTION,\n        query=embedding,\n        query_filter=Filter(must=[\n            FieldCondition(key=\"model\", match={\"value\": model}),\n            FieldCondition(key=\"expires_at\", range=Range(gte=time.time())),\n        ]),\n        limit=1,\n        score_threshold=SIMILARITY_THRESHOLD,\n    )\n    if results.points:\n        return results.points[0].payload[\"response\"]\n    return None\n\ndef semantic_cache_store(query: str, response: str, model: str):\n    embedding = embed_model.encode(query).tolist()\n    qdrant.upsert(\n        collection_name=CACHE_COLLECTION,\n        points=[PointStruct(\n            id=str(uuid.uuid4()),\n            vector=embedding,\n            payload={\n                \"query\": query,\n                \"response\": response,\n                \"model\": model,\n                \"created_at\": time.time(),\n                \"expires_at\": time.time() + CACHE_TTL_SECONDS,\n            },\n        )],\n    )\n\ndef smart_llm_call(query: str, model: str = \"gpt-4o-mini\") -> dict:\n    # 1. Semantic lookup\n    if cached_response := semantic_cache_lookup(query, model):\n        return {\"response\": cached_response, \"source\": \"semantic_cache\", \"cost\": 0}\n\n    # 2. LLM call\n    response = client.chat.completions.create(\n        model=model,\n        messages=[{\"role\": \"user\", \"content\": query}],\n    )\n    text = response.choices[0].message.content\n    cost = litellm.completion_cost(response)\n\n    # 3. Store in cache\n    semantic_cache_store(query, text, model)\n\n    return {\"response\": text, \"source\": \"llm_api\", \"cost\": cost}\n```\n\n## Layer 3: Provider-Side Prompt Caching\n\n```python\n# Anthropic — cache long system prompts (saves 90% on cached input tokens)\nimport anthropic\n\nclient = anthropic.Anthropic()\n\n# Long system prompt — mark for caching\nSYSTEM_PROMPT = open(\"knowledge-base.txt\").read()  # e.g., 50k tokens\n\ndef call_with_prompt_cache(user_question: str) -> str:\n    response = client.messages.create(\n        model=\"claude-sonnet-4-6\",\n        max_tokens=1024,\n        system=[\n            {\"type\": \"text\", \"text\": \"You are a helpful assistant.\"},\n            {\n                \"type\": \"text\",\n                \"text\": SYSTEM_PROMPT,\n                \"cache_control\": {\"type\": \"ephemeral\"},  # cache this block\n            }\n        ],\n        messages=[{\"role\": \"user\", \"content\": user_question}],\n    )\n    # Log cache efficiency\n    usage = response.usage\n    cache_savings = usage.cache_read_input_tokens * 0.9  # 90% discount on cached\n    print(f\"Cache hits: {usage.cache_read_input_tokens} tokens \"\n          f\"(saved ~${cache_savings * 3.0 \u002F 1_000_000:.4f})\")\n    return response.content[0].text\n\n# OpenAI — automatic for repeated prefixes (≥1,024 tokens)\n# No code change needed; cached tokens appear in usage.prompt_tokens_details\nresponse = client.chat.completions.create(\n    model=\"gpt-4o-mini\",\n    messages=[\n        {\"role\": \"system\", \"content\": LONG_SYSTEM_PROMPT},  # auto-cached\n        {\"role\": \"user\", \"content\": user_question},\n    ]\n)\ncached = response.usage.prompt_tokens_details.cached_tokens\nprint(f\"OpenAI cached {cached} tokens\")\n```\n\n## Cache Warming\n\n```python\nasync def warm_cache(common_queries: list[str], model: str):\n    \"\"\"Pre-populate cache with known frequent queries.\"\"\"\n    import asyncio\n    from openai import AsyncOpenAI\n\n    aclient = AsyncOpenAI()\n\n    async def warm_single(query: str):\n        if not semantic_cache_lookup(query, model):\n            response = await aclient.chat.completions.create(\n                model=model,\n                messages=[{\"role\": \"user\", \"content\": query}],\n            )\n            text = response.choices[0].message.content\n            semantic_cache_store(query, text, model)\n            print(f\"Warmed: {query[:50]}...\")\n\n    await asyncio.gather(*[warm_single(q) for q in common_queries])\n\n# Warm on startup\nimport asyncio\nasyncio.run(warm_cache(FREQUENT_QUERIES, \"gpt-4o-mini\"))\n```\n\n## Cache Metrics\n\n```python\nfrom prometheus_client import Counter, Histogram\n\ncache_hits = Counter(\"llm_cache_hits_total\", \"Cache hits\", [\"cache_layer\", \"model\"])\ncache_misses = Counter(\"llm_cache_misses_total\", \"Cache misses\", [\"model\"])\ncache_savings_usd = Counter(\"llm_cache_savings_usd_total\", \"USD saved by cache\", [\"model\"])\n\n# Use in your smart_llm_call function\nif source == \"semantic_cache\":\n    cache_hits.labels(cache_layer=\"semantic\", model=model).inc()\n    cache_savings_usd.labels(model=model).inc(estimated_cost)\nelse:\n    cache_misses.labels(model=model).inc()\n```\n\n## Redis Configuration for LLM Caching\n\n```bash\n# redis.conf tuning for LLM cache workload\nmaxmemory 8gb\nmaxmemory-policy allkeys-lru    # evict least-recently-used when full\nsave \"\"                          # disable persistence (cache is ephemeral)\nappendonly no\ntcp-keepalive 60\n```\n\n## Common Issues\n\n| Issue | Cause | Fix |\n|-------|-------|-----|\n| Low cache hit rate | Threshold too strict | Lower `SIMILARITY_THRESHOLD` to 0.82–0.85 |\n| Stale cached responses | Long TTL | Use topic-specific TTLs; invalidate on data updates |\n| Cache serving wrong answers | Threshold too loose | Raise threshold or add model-name filtering |\n| Redis OOM | No eviction policy | Set `maxmemory` + `allkeys-lru` |\n| Slow semantic lookup | Large cache collection | Add payload index on `model` + `expires_at` |\n\n## Best Practices\n\n- Start with exact cache — zero cost, instant wins for identical queries.\n- Semantic threshold of 0.88–0.92 balances hit rate vs. accuracy; tune with your data.\n- Set per-model TTLs: longer for stable knowledge (1 week), shorter for news\u002Fevents (1 hour).\n- Always filter by model name in semantic cache — different models give different answers.\n- Log cache hit rate as a KPI; target 30%+ for FAQ-style applications.\n\n## Related Skills\n\n- llm-cost-optimization (`llm-cost-optimization`) - Full cost strategy\n- llm-gateway (`llm-gateway`) - Gateway-level caching\n- vector-database-ops (`vector-database-ops`) - Qdrant setup\n- agent-observability (`agent-observability`) - Cache metrics dashboards\n\n## Limitations\n\n- Guidance executes against real environments: confirm target, blast radius, and rollback plan before applying anything.\n- Never deploy to production without explicit approval. Docs-only import: upstream scripts and templates not bundled.\n\n### Example\n\n```bash\ngit status && git diff --stat\nkubectl diff -f manifest.yaml\n```\n\n> Adapted from [BagelHole\u002FDevOps-Security-Agent-Skills](https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills) (MIT); frontmatter, When to Use\u002FLimitations, and safety boundaries added for upstream compliance. Docs-only import: helper scripts and templates not bundled.\n","# LLM Caching\n\nCut LLM costs and latency with exact match, semantic, and provider-side caching layers.\n\n## When to Use This Skill\n\nUse this skill when:\n- The same or similar queries are asked repeatedly (FAQ bots, support tools)\n- LLM API costs are growing and you need immediate savings\n- Serving high request volumes where repeated queries cause bottlenecks\n- Implementing prompt caching for long system prompts (Anthropic\u002FOpenAI)\n- Building offline-capable AI features that need response persistence\n\n## Caching Layers\n\n```\nRequest → Exact Cache → Semantic Cache → Provider Cache → LLM API\n             ↓ hit            ↓ hit             ↓ hit\n           instant          ~5ms           50-80% cheaper\n```\n\n## Layer 1: Exact Match Cache (Redis)\n\n```python\nimport hashlib\nimport json\nimport redis\nfrom openai import OpenAI\n\nr = redis.Redis(host=\"localhost\", port=6379, decode_responses=True)\nclient = OpenAI()\n\ndef build_cache_key(model: str, messages: list, temperature: float) -> str:\n    \"\"\"Deterministic key from request parameters.\"\"\"\n    payload = json.dumps({\n        \"model\": model,\n        \"messages\": messages,\n        \"temperature\": temperature,\n    }, sort_keys=True)\n    return f\"llm:exact:{hashlib.sha256(payload.encode()).hexdigest()}\"\n\ndef cached_completion(model: str, messages: list, temperature: float = 0.0,\n                      ttl: int = 3600) -> dict:\n    key = build_cache_key(model, messages, temperature)\n\n    # Check cache\n    if cached := r.get(key):\n        return json.loads(cached)\n\n    # Call API\n    response = client.chat.completions.create(\n        model=model, messages=messages, temperature=temperature\n    )\n    result = response.model_dump()\n\n    # Cache result (only cache deterministic responses)\n    if temperature == 0.0:\n        r.setex(key, ttl, json.dumps(result))\n\n    return result\n```\n\n## Layer 2: Semantic Cache (GPTCache)\n\n```python\nfrom gptcache import cache, Config\nfrom gptcache.adapter import openai\nfrom gptcache.embedding import Onnx\nfrom gptcache.manager import CacheBase, VectorBase, get_data_manager\nfrom gptcache.similarity_evaluation.distance import SearchDistanceEvaluation\n\n# Configure GPTCache with Qdrant backend\ndef init_gptcache(cache_obj, llm: str):\n    onnx = Onnx()                              # local embedding model\n    data_manager = get_data_manager(\n        CacheBase(\"redis\"),                    # metadata store\n        VectorBase(\"qdrant\",\n                   host=\"localhost\",\n                   port=6333,\n                   collection_name=f\"llm-cache-{llm}\",\n                   dimension=onnx.dimension),\n    )\n    cache_obj.init(\n        embedding_func=onnx.to_embeddings,\n        data_manager=data_manager,\n        similarity_evaluation=SearchDistanceEvaluation(),\n        config=Config(similarity_threshold=0.80),  # 80% similarity = cache hit\n    )\n\ncache.set_openai_key()\ninit_gptcache(cache, \"gpt-4o-mini\")\n\n# Now openai calls are automatically cached\nresponse = openai.ChatCompletion.create(\n    model=\"gpt-4o-mini\",\n    messages=[{\"role\": \"user\", \"content\": \"What is machine learning?\"}],\n)\n# Second call with similar question (\"Explain machine learning\") → cache hit\n```\n\n## Custom Semantic Cache (Production-Grade)\n\n```python\nfrom sentence_transformers import SentenceTransformer\nfrom qdrant_client import QdrantClient\nfrom qdrant_client.models import Distance, VectorParams, PointStruct, Filter, FieldCondition, Range\nimport numpy as np\nimport uuid\nimport time\n\nembed_model = SentenceTransformer(\"BAAI\u002Fbge-small-en-v1.5\")  # fast, 33M params\nqdrant = QdrantClient(\"http:\u002F\u002Flocalhost:6333\")\n\nCACHE_COLLECTION = \"semantic-cache\"\nSIMILARITY_THRESHOLD = 0.88\nCACHE_TTL_SECONDS = 86400  # 24h\n\n# Create collection once\nqdrant.create_collection(\n    collection_name=CACHE_COLLECTION,\n    vectors_config=VectorParams(size=384, distance=Distance.COSINE),\n    on_disk_payload=True,\n)\n\ndef semantic_cache_lookup(query: str, model: str) -> str | None:\n    embedding = embed_model.encode(query).tolist()\n    results = qdrant.query_points(\n        collection_name=CACHE_COLLECTION,\n        query=embedding,\n        query_filter=Filter(must=[\n            FieldCondition(key=\"model\", match={\"value\": model}),\n            FieldCondition(key=\"expires_at\", range=Range(gte=time.time())),\n        ]),\n        limit=1,\n        score_threshold=SIMILARITY_THRESHOLD,\n    )\n    if results.points:\n        return results.points[0].payload[\"response\"]\n    return None\n\ndef semantic_cache_store(query: str, response: str, model: str):\n    embedding = embed_model.encode(query).tolist()\n    qdrant.upsert(\n        collection_name=CACHE_COLLECTION,\n        points=[PointStruct(\n            id=str(uuid.uuid4()),\n            vector=embedding,\n            payload={\n                \"query\": query,\n                \"response\": response,\n                \"model\": model,\n                \"created_at\": time.time(),\n                \"expires_at\": time.time() + CACHE_TTL_SECONDS,\n            },\n        )],\n    )\n\ndef smart_llm_call(query: str, model: str = \"gpt-4o-mini\") -> dict:\n    # 1. Semantic lookup\n    if cached_response := semantic_cache_lookup(query, model):\n        return {\"response\": cached_response, \"source\": \"semantic_cache\", \"cost\": 0}\n\n    # 2. LLM call\n    response = client.chat.completions.create(\n        model=model,\n        messages=[{\"role\": \"user\", \"content\": query}],\n    )\n    text = response.choices[0].message.content\n    cost = litellm.completion_cost(response)\n\n    # 3. Store in cache\n    semantic_cache_store(query, text, model)\n\n    return {\"response\": text, \"source\": \"llm_api\", \"cost\": cost}\n```\n\n## Layer 3: Provider-Side Prompt Caching\n\n```python\n# Anthropic — cache long system prompts (saves 90% on cached input tokens)\nimport anthropic\n\nclient = anthropic.Anthropic()\n\n# Long system prompt — mark for caching\nSYSTEM_PROMPT = open(\"knowledge-base.txt\").read()  # e.g., 50k tokens\n\ndef call_with_prompt_cache(user_question: str) -> str:\n    response = client.messages.create(\n        model=\"claude-sonnet-4-6\",\n        max_tokens=1024,\n        system=[\n            {\"type\": \"text\", \"text\": \"You are a helpful assistant.\"},\n            {\n                \"type\": \"text\",\n                \"text\": SYSTEM_PROMPT,\n                \"cache_control\": {\"type\": \"ephemeral\"},  # cache this block\n            }\n        ],\n        messages=[{\"role\": \"user\", \"content\": user_question}],\n    )\n    # Log cache efficiency\n    usage = response.usage\n    cache_savings = usage.cache_read_input_tokens * 0.9  # 90% discount on cached\n    print(f\"Cache hits: {usage.cache_read_input_tokens} tokens \"\n          f\"(saved ~${cache_savings * 3.0 \u002F 1_000_000:.4f})\")\n    return response.content[0].text\n\n# OpenAI — automatic for repeated prefixes (≥1,024 tokens)\n# No code change needed; cached tokens appear in usage.prompt_tokens_details\nresponse = client.chat.completions.create(\n    model=\"gpt-4o-mini\",\n    messages=[\n        {\"role\": \"system\", \"content\": LONG_SYSTEM_PROMPT},  # auto-cached\n        {\"role\": \"user\", \"content\": user_question},\n    ]\n)\ncached = response.usage.prompt_tokens_details.cached_tokens\nprint(f\"OpenAI cached {cached} tokens\")\n```\n\n## Cache Warming\n\n```python\nasync def warm_cache(common_queries: list[str], model: str):\n    \"\"\"Pre-populate cache with known frequent queries.\"\"\"\n    import asyncio\n    from openai import AsyncOpenAI\n\n    aclient = AsyncOpenAI()\n\n    async def warm_single(query: str):\n        if not semantic_cache_lookup(query, model):\n            response = await aclient.chat.completions.create(\n                model=model,\n                messages=[{\"role\": \"user\", \"content\": query}],\n            )\n            text = response.choices[0].message.content\n            semantic_cache_store(query, text, model)\n            print(f\"Warmed: {query[:50]}...\")\n\n    await asyncio.gather(*[warm_single(q) for q in common_queries])\n\n# Warm on startup\nimport asyncio\nasyncio.run(warm_cache(FREQUENT_QUERIES, \"gpt-4o-mini\"))\n```\n\n## Cache Metrics\n\n```python\nfrom prometheus_client import Counter, Histogram\n\ncache_hits = Counter(\"llm_cache_hits_total\", \"Cache hits\", [\"cache_layer\", \"model\"])\ncache_misses = Counter(\"llm_cache_misses_total\", \"Cache misses\", [\"model\"])\ncache_savings_usd = Counter(\"llm_cache_savings_usd_total\", \"USD saved by cache\", [\"model\"])\n\n# Use in your smart_llm_call function\nif source == \"semantic_cache\":\n    cache_hits.labels(cache_layer=\"semantic\", model=model).inc()\n    cache_savings_usd.labels(model=model).inc(estimated_cost)\nelse:\n    cache_misses.labels(model=model).inc()\n```\n\n## Redis Configuration for LLM Caching\n\n```bash\n# redis.conf tuning for LLM cache workload\nmaxmemory 8gb\nmaxmemory-policy allkeys-lru    # evict least-recently-used when full\nsave \"\"                          # disable persistence (cache is ephemeral)\nappendonly no\ntcp-keepalive 60\n```\n\n## Common Issues\n\n| Issue | Cause | Fix |\n|-------|-------|-----|\n| Low cache hit rate | Threshold too strict | Lower `SIMILARITY_THRESHOLD` to 0.82–0.85 |\n| Stale cached responses | Long TTL | Use topic-specific TTLs; invalidate on data updates |\n| Cache serving wrong answers | Threshold too loose | Raise threshold or add model-name filtering |\n| Redis OOM | No eviction policy | Set `maxmemory` + `allkeys-lru` |\n| Slow semantic lookup | Large cache collection | Add payload index on `model` + `expires_at` |\n\n## Best Practices\n\n- Start with exact cache — zero cost, instant wins for identical queries.\n- Semantic threshold of 0.88–0.92 balances hit rate vs. accuracy; tune with your data.\n- Set per-model TTLs: longer for stable knowledge (1 week), shorter for news\u002Fevents (1 hour).\n- Always filter by model name in semantic cache — different models give different answers.\n- Log cache hit rate as a KPI; target 30%+ for FAQ-style applications.\n\n## Related Skills\n\n- llm-cost-optimization (`llm-cost-optimization`) - Full cost strategy\n- llm-gateway (`llm-gateway`) - Gateway-level caching\n- vector-database-ops (`vector-database-ops`) - Qdrant setup\n- agent-observability (`agent-observability`) - Cache metrics dashboards\n\n## Limitations\n\n- Guidance executes against real environments: confirm target, blast radius, and rollback plan before applying anything.\n- Never deploy to production without explicit approval. Docs-only import: upstream scripts and templates not bundled.\n\n### Example\n\n```bash\ngit status && git diff --stat\nkubectl diff -f manifest.yaml\n```\n\n> Adapted from [BagelHole\u002FDevOps-Security-Agent-Skills](https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills) (MIT); frontmatter, When to Use\u002FLimitations, and safety boundaries added for upstream compliance. Docs-only import: helper scripts and templates not bundled.\n",{"name":5,"risk":29,"source":48,"license":14,"category":49,"metadata":50,"date_added":53,"description":7,"source_repo":54,"source_type":32,"compatibility":55,"license_source":56},"https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills","devops",{"author":51,"version":52},"devops-skills","1.0","2026-09-20","BagelHole\u002FDevOps-Security-Agent-Skills","Requires the relevant platform CLIs (kubectl, helm, terraform, git, CI runners) and authorized access to the target environment. Docs-only; helper scripts and templates not bundled.","https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\u002Fblob\u002Fmain\u002FLICENSE","f3312ba749561b6159f477dbfc9313c875d40a56",".agents\u002Fskills_archive\u002Fllm-caching","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612","2026-10-25 23:50:58",{"source":8,"skillName":5,"commands":63,"prompt":73,"zipAvailable":74},[64,67,70],{"tool":65,"command":66},"npx","npx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill llm-caching",{"tool":68,"command":69},"bunx","bunx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill llm-caching",{"tool":71,"command":72},"pnpm","pnpm dlx skills add duc-duy-1612\u002FOnline-Delivery-Systems-Web --skill llm-caching","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fllm-caching\u002FSKILL.md",true,[],[77,98,122,147,175,199],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":86,"publishedAt":43},"01M6Q0FA5FZ410Z33Z3TFQ4DBA","zero-trust","Implement zero-trust network architecture. Configure identity-based access, micro-segmentation, and continuous verification. Use when implementing modern security architectures.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fzero-trust\u002FSKILL.md",[83],{"categoryNo":84,"categoryName":85,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000011","运维与部署",[87,88,89,90,91,92,79,93,37,36,94,34,32,31,95,29,96,28,27,97],"ZeRO","trust","zero trust","architecture","assessment","verification","segmentation","configure","active","scope","implementing",{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":43},"01M6Q0F1YZJW3KX5FBKEVZ8CV4","zcode-delegate","Delegate coding tasks to the Z.AI ZCode CLI only when the user explicitly requests it, while the orchestrator retains review and landing responsibility.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fzcode-delegate\u002FSKILL.md",[],[105,106,107,108,109,110,111,112,113,114,115,116,117,34,118,32,119,120,29,121],"delegate","zcode","zcode delegate","review","cli","orchestration","tasks","run","Bash","model","node","read","orchestrator","files","desktop app","app","agent-orchestration",{"skillNo":123,"name":124,"nameZh":6,"description":125,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":126,"categories":127,"keywords":128,"publishedAt":43},"01M6Q0ETK41VGNMYEXW0T00AJN","youtube-transcript","Fetch YouTube transcripts through DeepAPI or local fallback tooling and save clean text output.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-transcript\u002FSKILL.md",[],[129,130,131,132,133,134,135,136,137,138,139,140,141,32,31,142,143,144,145,146],"transcript","youtube transcript","youtube","safe","github","codex","research","license","https","text","fetch","tooling","clean","local","output","deepapi","youtube transcripts","blob",{"skillNo":148,"name":149,"nameZh":6,"description":150,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":151,"categories":152,"keywords":164,"publishedAt":43},"01M6Q0EPPH2PVMDW40017F8WX0","youtube-transcript-skills","Fetch YouTube video transcripts, search videos\u002Fchannels, browse channels, and extract playlists via the getyoutubetranscript.com API - free tier, no card required.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-transcript-skills\u002FSKILL.md",[153,156,159],{"categoryNo":154,"categoryName":155,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":157,"categoryName":158,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000016","图像与多媒体",{"categoryNo":160,"categoryName":161,"taxonomyType":162,"taxonomyTypeDesc":163},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[129,130,131,165,132,166,133,134,136,167,168,169,170,139,171,32,31,172,173,174],"youtube transcript skills","gemini","search","api","Video","browse","api-integration","free","card","channels",{"skillNo":176,"name":177,"nameZh":6,"description":178,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":179,"categories":180,"keywords":185,"publishedAt":43},"01M6Q0EAB96PX83W5PTE8A0R8D","youtube-notetaker","Turn YouTube talks into local study notes with slides, transcripts, editable annotations, and a markdown-backed viewer.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-notetaker\u002FSKILL.md",[181,184],{"categoryNo":182,"categoryName":183,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000003","演示文稿",{"categoryNo":157,"categoryName":158,"taxonomyType":20,"taxonomyTypeDesc":21},[131,186,187,109,132,188,189,190,191,136,169,192,193,194,195,31,196,142,197,198],"youtube notetaker","notetaker","workflow","official","notes","code","markdown","slides","annotations","README","study","dair-academy","viewer",{"skillNo":200,"name":201,"nameZh":6,"description":202,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":203,"categories":204,"keywords":206,"publishedAt":43},"01M6Q0E55F0GW5MX48V8HXS8B5","youtube-full","Fetch YouTube transcripts, search videos, browse channels, and extract playlists via TranscriptAPI — no yt-dlp, no Google API key, works from any cloud server.","https:\u002F\u002Fgithub.com\u002Fduc-duy-1612\u002FOnline-Delivery-Systems-Web\u002Fblob\u002Fmain\u002F.agents\u002Fskills_archive\u002Fyoutube-full\u002FSKILL.md",[205],{"categoryNo":154,"categoryName":155,"taxonomyType":20,"taxonomyTypeDesc":21},[207,131,208,209,132,166,167,210,211,168,169,170,139,212,171,32,31,172,213,145],"youtube full","full","google","server","dlp","credits","automatic"]