[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M73DHNGVFM75B2DFEYREKQWT":3,"skill-same:01M73DHNGVFM75B2DFEYREKQWT":47},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":23,"commitSha":24,"skillDirectory":25,"forksCount":26,"repositoryHtmlUrl":27,"repositoryHomepageUrl":28,"ownerHtmlUrl":29,"ownerBlogUrl":30,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M73DHNGVFM75B2DFEYREKQWT","llama-cpp-quantization",null,"Horonomos-laps\u002FYana-AI","Horonomos-laps","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F355299122?v=4","01M76B3QPXE88ET2SQPDPVATZT"," Horonomos-laps","01M72ZQRJPCYPSSWT4D00VJR2A",13,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fllama-cpp-quantization\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: llama-cpp-quantization\ndescription: llama.cpp GGUF\u002FGGML quantization patterns for local LLM inference. Quantization levels (Q4_K_M, Q8_0, F16), CPU\u002FGPU offloading, context window sizing, and embedding extraction via llama.cpp HTTP server. Sources: ggerganov\u002Fllama.cpp (MIT).\norigin: yana-ai — synthesized from ggerganov\u002Fllama.cpp (MIT)\nlicense: Apache-2.0\nversion: 1.0.0\ncompatibility: yana-ai >= 1.3.51\n---\n\n# \u002Fllama-cpp-quantization\n\n## When to Use\n\n- Run open-source LLMs locally without GPU (CPU inference via GGUF)\n- Choose quantization level based on quality\u002Fmemory tradeoff\n- Embed llama.cpp HTTP server for OpenAI-compatible local API\n- Extract embeddings from local model for [[in-memory-vector-storage]]\n\n## Do NOT use for\n\n- Cloud LLM APIs (Anthropic\u002FOpenAI — use [[http-client-auth-patterns]])\n- Production GPU clusters (use [[vllm-paged-attention]] or [[tgi-streaming-inference]])\n\n---\n\n## Quantization level guide\n\n```\nFormat      Bits  Size(7B)  Quality  Use case\n──────────────────────────────────────────────\nQ2_K        2.6   2.8 GB    poor     emergency low-RAM only\nQ4_K_S      4.4   4.1 GB    ok       minimum usable\nQ4_K_M      4.8   4.5 GB    good     ← default recommendation\nQ5_K_M      5.7   5.3 GB    great    if RAM allows\nQ8_0        8.0   7.2 GB    near-lossless  pre-production\nF16        16.0  13.5 GB    lossless  fine-tuning only\n\nRule: Q4_K_M for dev\u002Ftesting, Q8_0 for production inference gates\n```\n\n---\n\n## Start llama.cpp HTTP server\n\n```bash\n# Download GGUF model\ncurl -L -o models\u002Fmistral-7b-q4.gguf \\\n  \"https:\u002F\u002Fhuggingface.co\u002FTheBloke\u002FMistral-7B-v0.1-GGUF\u002Fresolve\u002Fmain\u002Fmistral-7b-v0.1.Q4_K_M.gguf\"\n\n# Start server (OpenAI-compatible API)\n.\u002Fllama-server \\\n  --model    models\u002Fmistral-7b-q4.gguf \\\n  --ctx-size 4096       \\   # context window (tokens)\n  --n-gpu-layers 35     \\   # layers on GPU (0 = CPU-only)\n  --threads  8          \\   # CPU threads\n  --port     8080       \\\n  --host     0.0.0.0    \\\n  --parallel 4              # concurrent request slots\n```\n\n---\n\n## OpenAI-compatible client call\n\n```javascript\nimport OpenAI from 'openai'\n\nconst client = new OpenAI({\n  baseURL: 'http:\u002F\u002Flocalhost:8080\u002Fv1',\n  apiKey:  'not-needed',   \u002F\u002F llama.cpp ignores this\n})\n\nconst completion = await client.chat.completions.create({\n  model:      'local',\n  messages:   [{ role: 'user', content: 'What is 2+2?' }],\n  max_tokens: 256,\n  temperature: 0.1,\n  stream:     true,\n})\n\nfor await (const chunk of completion) {\n  process.stdout.write(chunk.choices[0]?.delta?.content ?? '')\n}\n```\n\n---\n\n## Quantize a model yourself\n\n```bash\n# Convert HuggingFace model → GGUF\npython3 convert_hf_to_gguf.py \\\n  --outfile models\u002Fmy-model-f16.gguf \\\n  --model   \u002Fpath\u002Fto\u002Fhf-model\n\n# Quantize F16 → Q4_K_M\n.\u002Fquantize \\\n  models\u002Fmy-model-f16.gguf \\\n  models\u002Fmy-model-q4km.gguf \\\n  Q4_K_M\n```\n\n---\n\n## Extract embeddings\n\n```bash\n# GET \u002Fembedding (llama.cpp server endpoint)\ncurl -s http:\u002F\u002Flocalhost:8080\u002Fembedding \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"content\": \"agent identity for vector search\"}' \\\n  | jq '.embedding | length'   # → 4096 (model dim)\n```\n\n---\n\n## Anti-Fake-Pass Checklist\n\n```\n❌ ctx-size > model's trained max → gibberish past context boundary (check model card)\n❌ n-gpu-layers too high → VRAM OOM; start at 0 and increment until it fits\n❌ Q2_K for code tasks → quality too low; code fails to compile, logic errors\n❌ --parallel > available RAM\u002FVRAM → OOM on concurrent requests\n❌ F16 for CPU inference → 2× slower than Q8_0 with same quality; use Q8_0\n❌ Not setting --threads to nproc → defaults to 1 thread, 10× slower\n```\n","# \u002Fllama-cpp-quantization\n\n## When to Use\n\n- Run open-source LLMs locally without GPU (CPU inference via GGUF)\n- Choose quantization level based on quality\u002Fmemory tradeoff\n- Embed llama.cpp HTTP server for OpenAI-compatible local API\n- Extract embeddings from local model for [[in-memory-vector-storage]]\n\n## Do NOT use for\n\n- Cloud LLM APIs (Anthropic\u002FOpenAI — use [[http-client-auth-patterns]])\n- Production GPU clusters (use [[vllm-paged-attention]] or [[tgi-streaming-inference]])\n\n---\n\n## Quantization level guide\n\n```\nFormat      Bits  Size(7B)  Quality  Use case\n──────────────────────────────────────────────\nQ2_K        2.6   2.8 GB    poor     emergency low-RAM only\nQ4_K_S      4.4   4.1 GB    ok       minimum usable\nQ4_K_M      4.8   4.5 GB    good     ← default recommendation\nQ5_K_M      5.7   5.3 GB    great    if RAM allows\nQ8_0        8.0   7.2 GB    near-lossless  pre-production\nF16        16.0  13.5 GB    lossless  fine-tuning only\n\nRule: Q4_K_M for dev\u002Ftesting, Q8_0 for production inference gates\n```\n\n---\n\n## Start llama.cpp HTTP server\n\n```bash\n# Download GGUF model\ncurl -L -o models\u002Fmistral-7b-q4.gguf \\\n  \"https:\u002F\u002Fhuggingface.co\u002FTheBloke\u002FMistral-7B-v0.1-GGUF\u002Fresolve\u002Fmain\u002Fmistral-7b-v0.1.Q4_K_M.gguf\"\n\n# Start server (OpenAI-compatible API)\n.\u002Fllama-server \\\n  --model    models\u002Fmistral-7b-q4.gguf \\\n  --ctx-size 4096       \\   # context window (tokens)\n  --n-gpu-layers 35     \\   # layers on GPU (0 = CPU-only)\n  --threads  8          \\   # CPU threads\n  --port     8080       \\\n  --host     0.0.0.0    \\\n  --parallel 4              # concurrent request slots\n```\n\n---\n\n## OpenAI-compatible client call\n\n```javascript\nimport OpenAI from 'openai'\n\nconst client = new OpenAI({\n  baseURL: 'http:\u002F\u002Flocalhost:8080\u002Fv1',\n  apiKey:  'not-needed',   \u002F\u002F llama.cpp ignores this\n})\n\nconst completion = await client.chat.completions.create({\n  model:      'local',\n  messages:   [{ role: 'user', content: 'What is 2+2?' }],\n  max_tokens: 256,\n  temperature: 0.1,\n  stream:     true,\n})\n\nfor await (const chunk of completion) {\n  process.stdout.write(chunk.choices[0]?.delta?.content ?? '')\n}\n```\n\n---\n\n## Quantize a model yourself\n\n```bash\n# Convert HuggingFace model → GGUF\npython3 convert_hf_to_gguf.py \\\n  --outfile models\u002Fmy-model-f16.gguf \\\n  --model   \u002Fpath\u002Fto\u002Fhf-model\n\n# Quantize F16 → Q4_K_M\n.\u002Fquantize \\\n  models\u002Fmy-model-f16.gguf \\\n  models\u002Fmy-model-q4km.gguf \\\n  Q4_K_M\n```\n\n---\n\n## Extract embeddings\n\n```bash\n# GET \u002Fembedding (llama.cpp server endpoint)\ncurl -s http:\u002F\u002Flocalhost:8080\u002Fembedding \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"content\": \"agent identity for vector search\"}' \\\n  | jq '.embedding | length'   # → 4096 (model dim)\n```\n\n---\n\n## Anti-Fake-Pass Checklist\n\n```\n❌ ctx-size > model's trained max → gibberish past context boundary (check model card)\n❌ n-gpu-layers too high → VRAM OOM; start at 0 and increment until it fits\n❌ Q2_K for code tasks → quality too low; code fails to compile, logic errors\n❌ --parallel > available RAM\u002FVRAM → OOM on concurrent requests\n❌ F16 for CPU inference → 2× slower than Q8_0 with same quality; use Q8_0\n❌ Not setting --threads to nproc → defaults to 1 thread, 10× slower\n```\n",{},"main","481dcd7dd2e288b9d627a2bd017db70e64e6c6fa","core\u002Fskills\u002Fllama-cpp-quantization",2,"https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI","https:\u002F\u002Fyana.vutam.link\u002F","https:\u002F\u002Fgithub.com\u002FHoronomos-laps","https:\u002F\u002Fyana.vutam.link","An open-source governed runtime & execution environment for multi-agent AI systems.","2026-11-14 13:24:09",{"source":7,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add Horonomos-laps\u002FYana-AI --skill llama-cpp-quantization",{"tool":39,"command":40},"bunx","bunx skills add Horonomos-laps\u002FYana-AI --skill llama-cpp-quantization",{"tool":42,"command":43},"pnpm","pnpm dlx skills add Horonomos-laps\u002FYana-AI --skill llama-cpp-quantization","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fllama-cpp-quantization\u002FSKILL.md",true,[],[48,56,63,71,79,87],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":52,"licenseFileUrl":6,"sourceHtmlUrl":53,"categories":54,"keywords":55,"publishedAt":18},"01M735GKET5YVYKAY62W2P5YXB","openai--sentry--sentry","Use when the user asks to inspect Sentry issues or events, summarize recent production errors, or pull basic Sentry health data via the Sentry API; perform read-only queries with the bundled script and require `SENTRY_AUTH_TOKEN`.","MIT","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fopenai--sentry--sentry\u002FSKILL.md",[],[],{"skillNo":57,"name":58,"nameZh":6,"description":59,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":18},"01M73DCAP83T17HY7SMGK7MGFE","ito-basket-compare","Compare Itô prediction-market baskets against a user's knowledge base, portfolio notes, financial context, watchlist, or research thesis. Use for read-only basket comparison and gap analysis without investment advice or live trading.","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fito-basket-compare\u002FSKILL.md",[],[],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":67,"sourceHtmlUrl":68,"categories":69,"keywords":70,"publishedAt":18},"01M734CYEMZFDR3KH6SW0JP8TD","implementing-network-traffic-baselining","Build network traffic baselines from NetFlow\u002FIPFIX data using Python pandas for statistical analysis, z-score anomaly detection, and hourly\u002Fdaily traffic pattern profiling","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-network-traffic-baselining\u002FSKILL.md",[],[],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":75,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":18},"01M73ASZ6NJRJM44Z2P84NGQ3Q","detecting-evasion-techniques-in-endpoint-logs","Detects defense evasion techniques used by adversaries in endpoint logs including log tampering, timestomping, process injection, and security tool disabling. Use when investigating suspicious endpoint behavior, building detection rules for evasion tactics, or conducting threat hunting for stealthy adversary activity. Activates for requests involving evasion detection, defense evasion analysis, log tampering detection, or MITRE ATT&CK TA0005.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002Fcore\u002Fskills\u002Fdetecting-evasion-techniques-in-endpoint-logs\u002FSKILL.md",[],[],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":83,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":18},"01M734AFNXD06Z78QRK0VKGT7Q","implementing-microsegmentation-with-guardicore","Implementing microsegmentation using Akamai Guardicore Segmentation to map application dependencies, create granular network policies, visualize east-west traffic flows, and enforce least-privilege communication between workloads across data centers and cloud.","https:\u002F\u002Fgithub.com\u002Fyanacuti1121\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fimplementing-microsegmentation-with-guardicore\u002FSKILL.md",[],[],{"skillNo":88,"name":89,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":18},"01M736TK27QMCXK0DVCH25SJN5","ramdisk-tmpfs-patterns","https:\u002F\u002Fgithub.com\u002FHoronomos-laps\u002FYana-AI\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Framdisk-tmpfs-patterns\u002FSKILL.md",[],[]]