[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M71JBFPGWV2QDKA54XC4QQHD":3,"skill-same:01M71JBFPGWV2QDKA54XC4QQHD":97},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":62,"commitSha":63,"skillDirectory":64,"forksCount":65,"repositoryHtmlUrl":66,"repositoryHomepageUrl":6,"ownerHtmlUrl":67,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":68,"install":69,"resources":82},"01M71JBFPGWV2QDKA54XC4QQHD","llama-cpp",null,"Run LLM inference with llama.cpp on CPU, Apple Silicon, AMD\u002FIntel GPUs, or NVIDIA — plus GGUF model conversion and quantization (2–8 bit with K-quants and imatrix). Covers CLI, Python bindings, OpenAI-compatible server, and Ollama\u002FLM Studio integration. Use for edge deployment, M1\u002FM2\u002FM3\u002FM4 Macs, CUDA-less environments, or flexible local quantization.","xuanxuan1983\u002Fhermes-skills-backup","xuanxuan1983","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F151701653?v=4","01M71HCZNB5H19QJ6YBF98W2WH","01M71HCZP14T0921BD0TC9EPD3",1,"MIT","https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Finference\u002Fllama-cpp\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",[23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"llama cpp","Llama","cpp","Model Compression","CPU Inference","llama.cpp","GGUF","lm studio","silicon","run","CPU","cli","ollama","server","apple","conversion","studio","local","model","bit","2026-11-10 10:12:30",false,"---\nname: llama-cpp\ndescription: Run LLM inference with llama.cpp on CPU, Apple Silicon, AMD\u002FIntel GPUs, or NVIDIA — plus GGUF model conversion and quantization (2–8 bit with K-quants and imatrix). Covers CLI, Python bindings, OpenAI-compatible server, and Ollama\u002FLM Studio integration. Use for edge deployment, M1\u002FM2\u002FM3\u002FM4 Macs, CUDA-less environments, or flexible local quantization.\nversion: 2.0.0\nauthor: Orchestra Research\nlicense: MIT\ndependencies: [llama-cpp-python>=0.2.0]\nmetadata:\n  hermes:\n    tags: [llama.cpp, GGUF, Quantization, CPU Inference, Apple Silicon, Edge Deployment, Non-NVIDIA, AMD GPUs, Intel GPUs, Embedded, Model Compression]\n---\n\n# llama.cpp + GGUF\n\nPure C\u002FC++ LLM inference with minimal dependencies, plus the GGUF (GPT-Generated Unified Format) standard used for quantized weights. One toolchain covers conversion, quantization, and serving.\n\n## When to use\n\n**Use llama.cpp + GGUF when:**\n- Running on CPU-only machines or Apple Silicon (M1\u002FM2\u002FM3\u002FM4) with Metal acceleration\n- Using AMD (ROCm) or Intel GPUs where CUDA isn't available\n- Edge deployment (Raspberry Pi, embedded systems, consumer laptops)\n- Need flexible quantization (2–8 bit with K-quants)\n- Want local AI tools (LM Studio, Ollama, text-generation-webui, koboldcpp)\n- Want a single binary deploy without Docker\u002FPython\n\n**Key advantages:**\n- Universal hardware: CPU, Apple Silicon, NVIDIA, AMD, Intel\n- No Python runtime required (pure C\u002FC++)\n- K-quants + imatrix for better low-bit quality\n- OpenAI-compatible server built in\n- Rich ecosystem (Ollama, LM Studio, llama-cpp-python)\n\n**Use alternatives instead:**\n- **vLLM** — NVIDIA GPUs, PagedAttention, Python-first, max throughput\n- **TensorRT-LLM** — Production NVIDIA (A100\u002FH100), maximum speed\n- **AWQ\u002FGPTQ** — Calibrated quantization for NVIDIA-only deployments\n- **bitsandbytes** — Simple HuggingFace transformers integration\n- **HQQ** — Fast calibration-free quantization\n\n## Quick start\n\n### Install\n\n```bash\n# macOS \u002F Linux (simplest)\nbrew install llama.cpp\n\n# Or build from source\ngit clone https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\ncd llama.cpp\nmake                        # CPU\nmake GGML_METAL=1           # Apple Silicon\nmake GGML_CUDA=1            # NVIDIA CUDA\nmake LLAMA_HIP=1            # AMD ROCm\n\n# Python bindings (optional)\npip install llama-cpp-python\n# With CUDA:   CMAKE_ARGS=\"-DGGML_CUDA=on\" pip install llama-cpp-python --force-reinstall --no-cache-dir\n# With Metal:  CMAKE_ARGS=\"-DGGML_METAL=on\" pip install llama-cpp-python --force-reinstall --no-cache-dir\n```\n\n### Download a pre-quantized GGUF\n\n```bash\n# TheBloke hosts most popular models pre-quantized\nhuggingface-cli download \\\n    TheBloke\u002FLlama-2-7B-Chat-GGUF \\\n    llama-2-7b-chat.Q4_K_M.gguf \\\n    --local-dir models\u002F\n```\n\n### Or convert a HuggingFace model to GGUF\n\n```bash\n# 1. Download HF model\nhuggingface-cli download meta-llama\u002FLlama-3.1-8B --local-dir .\u002Fllama-3.1-8b\n\n# 2. Convert to FP16 GGUF\npython convert_hf_to_gguf.py .\u002Fllama-3.1-8b \\\n    --outfile llama-3.1-8b-f16.gguf \\\n    --outtype f16\n\n# 3. Quantize to Q4_K_M\n.\u002Fllama-quantize llama-3.1-8b-f16.gguf llama-3.1-8b-q4_k_m.gguf Q4_K_M\n```\n\n### Run inference\n\n```bash\n# One-shot prompt\n.\u002Fllama-cli -m model.Q4_K_M.gguf -p \"Explain quantum computing\" -n 256\n\n# Interactive chat\n.\u002Fllama-cli -m model.Q4_K_M.gguf --interactive\n\n# With GPU offload\n.\u002Fllama-cli -m model.Q4_K_M.gguf -ngl 35 -p \"Hello!\"\n```\n\n### Serve an OpenAI-compatible API\n\n```bash\n.\u002Fllama-server \\\n    -m model.Q4_K_M.gguf \\\n    --host 0.0.0.0 \\\n    --port 8080 \\\n    -ngl 35 \\\n    -c 4096 \\\n    --parallel 4 \\\n    --cont-batching\n```\n\n```bash\ncurl http:\u002F\u002Flocalhost:8080\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"local\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}],\n    \"temperature\": 0.7,\n    \"max_tokens\": 100\n  }'\n```\n\n## Quantization formats (GGUF)\n\n### K-quant methods (recommended)\n\n| Type | Bits | Size (7B) | Quality | Use Case |\n|------|------|-----------|---------|----------|\n| Q2_K | 2.5 | ~2.8 GB | Low | Extreme compression (testing only) |\n| Q3_K_S | 3.0 | ~3.0 GB | Low-Med | Memory constrained |\n| Q3_K_M | 3.3 | ~3.3 GB | Medium | Fits small devices |\n| Q4_K_S | 4.0 | ~3.8 GB | Med-High | Speed critical |\n| **Q4_K_M** | 4.5 | ~4.1 GB | High | **Recommended default** |\n| Q5_K_S | 5.0 | ~4.6 GB | High | Quality focused |\n| Q5_K_M | 5.5 | ~4.8 GB | Very High | High quality |\n| Q6_K | 6.0 | ~5.5 GB | Excellent | Near-original |\n| Q8_0 | 8.0 | ~7.2 GB | Best | Maximum quality, minimal degradation |\n\n**Variant suffixes** — `_S` (Small, faster, lower quality), `_M` (Medium, balanced), `_L` (Large, better quality).\n\n**Legacy (Q4_0\u002FQ4_1\u002FQ5_0\u002FQ5_1) exist** but always prefer K-quants for better quality\u002Fsize ratio.\n\n**IQ quantization** — ultra-low-bit with importance-aware methods: IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_XS, IQ3_S, IQ4_XS. Require `--imatrix`.\n\n**Task-specific defaults:**\n- General chat \u002F assistants: Q4_K_M, or Q5_K_M if RAM allows\n- Code generation: Q5_K_M or Q6_K (higher precision helps)\n- Technical \u002F medical: Q6_K or Q8_0\n- Very large (70B, 405B) on consumer hardware: Q3_K_M or Q4_K_S\n- Raspberry Pi \u002F edge: Q2_K or Q3_K_S\n\n## Conversion workflows\n\n### Basic: HF → GGUF → quantized\n\n```bash\npython convert_hf_to_gguf.py .\u002Fmodel --outfile model-f16.gguf --outtype f16\n.\u002Fllama-quantize model-f16.gguf model-q4_k_m.gguf Q4_K_M\n.\u002Fllama-cli -m model-q4_k_m.gguf -p \"Hello!\" -n 50\n```\n\n### With importance matrix (imatrix) — better low-bit quality\n\n`imatrix` gives 10–20% perplexity improvement at Q4, essential at Q3 and below.\n\n```bash\n# 1. Convert to FP16 GGUF\npython convert_hf_to_gguf.py .\u002Fmodel --outfile model-f16.gguf\n\n# 2. Prepare calibration data (diverse text, ~100MB is ideal)\ncat > calibration.txt \u003C\u003C 'EOF'\nThe quick brown fox jumps over the lazy dog.\nMachine learning is a subset of artificial intelligence.\n# Add more diverse text samples...\nEOF\n\n# 3. Generate importance matrix\n.\u002Fllama-imatrix -m model-f16.gguf \\\n    -f calibration.txt \\\n    --chunk 512 \\\n    -o model.imatrix \\\n    -ngl 35\n\n# 4. Quantize with imatrix\n.\u002Fllama-quantize --imatrix model.imatrix \\\n    model-f16.gguf model-q4_k_m.gguf Q4_K_M\n```\n\n### Multi-quant batch\n\n```bash\n#!\u002Fbin\u002Fbash\nMODEL=\"llama-3.1-8b-f16.gguf\"\nIMATRIX=\"llama-3.1-8b.imatrix\"\n\n.\u002Fllama-imatrix -m $MODEL -f wiki.txt -o $IMATRIX -ngl 35\n\nfor QUANT in Q4_K_M Q5_K_M Q6_K Q8_0; do\n    OUTPUT=\"llama-3.1-8b-${QUANT,,}.gguf\"\n    .\u002Fllama-quantize --imatrix $IMATRIX $MODEL $OUTPUT $QUANT\n    echo \"Created: $OUTPUT ($(du -h $OUTPUT | cut -f1))\"\ndone\n```\n\n### Quality testing (perplexity)\n\n```bash\n.\u002Fllama-perplexity -m model.gguf -f wikitext-2-raw\u002Fwiki.test.raw -c 512\n# Baseline FP16: ~5.96  |  Q4_K_M: ~6.06 (+1.7%)  |  Q2_K: ~6.87 (+15.3%)\n```\n\n## Python bindings (llama-cpp-python)\n\n### Basic generation\n\n```python\nfrom llama_cpp import Llama\n\nllm = Llama(\n    model_path=\".\u002Fmodel-q4_k_m.gguf\",\n    n_ctx=4096,\n    n_gpu_layers=35,     # 0 for CPU only, 99 to offload everything\n    n_threads=8,\n)\n\noutput = llm(\n    \"What is machine learning?\",\n    max_tokens=256,\n    temperature=0.7,\n    stop=[\"\u003C\u002Fs>\", \"\\n\\n\"],\n)\nprint(output[\"choices\"][0][\"text\"])\n```\n\n### Chat completion + streaming\n\n```python\nllm = Llama(\n    model_path=\".\u002Fmodel-q4_k_m.gguf\",\n    n_ctx=4096,\n    n_gpu_layers=35,\n    chat_format=\"llama-3\",    # Or \"chatml\", \"mistral\", etc.\n)\n\n# Non-streaming\nresponse = llm.create_chat_completion(\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"What is Python?\"},\n    ],\n    max_tokens=256,\n    temperature=0.7,\n)\nprint(response[\"choices\"][0][\"message\"][\"content\"])\n\n# Streaming\nfor chunk in llm(\"Explain quantum computing:\", max_tokens=256, stream=True):\n    print(chunk[\"choices\"][0][\"text\"], end=\"\", flush=True)\n```\n\n### Embeddings\n\n```python\nllm = Llama(model_path=\".\u002Fmodel-q4_k_m.gguf\", embedding=True, n_gpu_layers=35)\nvec = llm.embed(\"This is a test sentence.\")\nprint(f\"Embedding dimension: {len(vec)}\")\n```\n\n## Hardware acceleration\n\n### Apple Silicon (Metal)\n\n```bash\nmake clean && make GGML_METAL=1\n.\u002Fllama-cli -m model.gguf -ngl 99 -p \"Hello\"   # offload all layers\n```\n\n```python\nllm = Llama(\n    model_path=\"model.gguf\",\n    n_gpu_layers=99,     # Offload everything\n    n_threads=1,         # Metal handles parallelism\n)\n```\n\nPerformance: M3 Max ~40–60 tok\u002Fs on Llama 2-7B Q4_K_M.\n\n### NVIDIA (CUDA)\n\n```bash\nmake clean && make GGML_CUDA=1\n.\u002Fllama-cli -m model.gguf -ngl 35 -p \"Hello\"\n\n# Hybrid for large models\n.\u002Fllama-cli -m llama-70b.Q4_K_M.gguf -ngl 20   # GPU: 20 layers, CPU: rest\n\n# Multi-GPU split\n.\u002Fllama-cli -m large-model.gguf --tensor-split 0.5,0.5 -ngl 60\n```\n\n### AMD (ROCm)\n\n```bash\nmake LLAMA_HIP=1\n.\u002Fllama-cli -m model.gguf -ngl 999\n```\n\n### CPU\n\n```bash\n# Match PHYSICAL cores, not logical\n.\u002Fllama-cli -m model.gguf -t 8 -p \"Hello\"\n\n# BLAS acceleration (2–3× speedup)\nmake LLAMA_OPENBLAS=1\n```\n\n```python\nllm = Llama(\n    model_path=\"model.gguf\",\n    n_gpu_layers=0,\n    n_threads=8,\n    n_batch=512,         # Larger batch = faster prompt processing\n)\n```\n\n## Performance benchmarks\n\n### CPU (Llama 2-7B Q4_K_M)\n\n| CPU | Threads | Speed |\n|-----|---------|-------|\n| Apple M3 Max (Metal) | 16 | 50 tok\u002Fs |\n| AMD Ryzen 9 7950X | 32 | 35 tok\u002Fs |\n| Intel i9-13900K | 32 | 30 tok\u002Fs |\n\n### GPU offloading on RTX 4090\n\n| Layers GPU | Speed | VRAM |\n|------------|-------|------|\n| 0 (CPU only) | 30 tok\u002Fs | 0 GB |\n| 20 (hybrid) | 80 tok\u002Fs | 8 GB |\n| 35 (all) | 120 tok\u002Fs | 12 GB |\n\n## Supported models\n\n- **LLaMA family**: Llama 2 (7B\u002F13B\u002F70B), Llama 3 (8B\u002F70B\u002F405B), Code Llama\n- **Mistral family**: Mistral 7B, Mixtral 8x7B\u002F8x22B\n- **Other**: Falcon, BLOOM, GPT-J, Phi-3, Gemma, Qwen, LLaVA (vision), Whisper (audio)\n\nFind GGUF models: https:\u002F\u002Fhuggingface.co\u002Fmodels?library=gguf\n\n## Ecosystem integrations\n\n### Ollama\n\n```bash\ncat > Modelfile \u003C\u003C 'EOF'\nFROM .\u002Fmodel-q4_k_m.gguf\nTEMPLATE \"\"\"{{ .System }}\n{{ .Prompt }}\"\"\"\nPARAMETER temperature 0.7\nPARAMETER num_ctx 4096\nEOF\n\nollama create mymodel -f Modelfile\nollama run mymodel \"Hello!\"\n```\n\n### LM Studio\n\n1. Place GGUF file in `~\u002F.cache\u002Flm-studio\u002Fmodels\u002F`\n2. Open LM Studio and select the model\n3. Configure context length and GPU offload, start inference\n\n### text-generation-webui\n\n```bash\ncp model-q4_k_m.gguf text-generation-webui\u002Fmodels\u002F\npython server.py --model model-q4_k_m.gguf --loader llama.cpp --n-gpu-layers 35\n```\n\n### OpenAI client → llama-server\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http:\u002F\u002Flocalhost:8080\u002Fv1\", api_key=\"not-needed\")\nresponse = client.chat.completions.create(\n    model=\"local-model\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n    max_tokens=256,\n)\nprint(response.choices[0].message.content)\n```\n\n## Best practices\n\n1. **Use K-quants** — Q4_K_M is the recommended default\n2. **Use imatrix** for Q4 and below (calibration improves quality substantially)\n3. **Offload as many layers as VRAM allows** — start high, reduce by 5 on OOM\n4. **Thread count** — match physical cores, not logical\n5. **Batch size** — increase `n_batch` (e.g. 512) for faster prompt processing\n6. **Context** — start at 4096, grow only as needed (memory scales with ctx)\n7. **Flash Attention** — add `--flash-attn` if your build supports it\n\n## Common issues (quick fixes)\n\n**Model loads slowly** — use `--mmap` for memory-mapped loading.\n\n**Out of memory (GPU)** — reduce `-ngl`, use a smaller quant (Q4_K_S \u002F Q3_K_M), or quantize the KV cache:\n```python\nLlama(model_path=\"...\", type_k=2, type_v=2, n_gpu_layers=35)  # Q4_0 KV cache\n```\n\n**Garbage output** — wrong `chat_format`, temperature too high, or model file corrupted. Test with `temperature=0.1` and verify FP16 baseline works.\n\n**Connection refused (server)** — bind to `--host 0.0.0.0`, check `lsof -i :8080`.\n\nSee `references\u002Ftroubleshooting.md` for the full playbook.\n\n## References\n\n- **[advanced-usage.md](references\u002Fadvanced-usage.md)** — speculative decoding, batched inference, grammar-constrained generation, LoRA, multi-GPU, custom builds, benchmark scripts\n- **[quantization.md](references\u002Fquantization.md)** — perplexity tables, use-case guide, model size scaling (7B\u002F13B\u002F70B RAM needs), imatrix deep dive\n- **[server.md](references\u002Fserver.md)** — OpenAI API endpoints, Docker deployment, NGINX load balancing, monitoring\n- **[optimization.md](references\u002Foptimization.md)** — CPU threading, BLAS, GPU offload heuristics, batch tuning, benchmarks\n- **[troubleshooting.md](references\u002Ftroubleshooting.md)** — install\u002Fconvert\u002Fquantize\u002Finference\u002Fserver issues, Apple Silicon, debugging\n\n## Resources\n\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\n- **Python bindings**: https:\u002F\u002Fgithub.com\u002Fabetlen\u002Fllama-cpp-python\n- **Pre-quantized models**: https:\u002F\u002Fhuggingface.co\u002FTheBloke\n- **GGUF converter Space**: https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fggml-org\u002Fgguf-my-repo\n- **License**: MIT\n","# llama.cpp + GGUF\n\nPure C\u002FC++ LLM inference with minimal dependencies, plus the GGUF (GPT-Generated Unified Format) standard used for quantized weights. One toolchain covers conversion, quantization, and serving.\n\n## When to use\n\n**Use llama.cpp + GGUF when:**\n- Running on CPU-only machines or Apple Silicon (M1\u002FM2\u002FM3\u002FM4) with Metal acceleration\n- Using AMD (ROCm) or Intel GPUs where CUDA isn't available\n- Edge deployment (Raspberry Pi, embedded systems, consumer laptops)\n- Need flexible quantization (2–8 bit with K-quants)\n- Want local AI tools (LM Studio, Ollama, text-generation-webui, koboldcpp)\n- Want a single binary deploy without Docker\u002FPython\n\n**Key advantages:**\n- Universal hardware: CPU, Apple Silicon, NVIDIA, AMD, Intel\n- No Python runtime required (pure C\u002FC++)\n- K-quants + imatrix for better low-bit quality\n- OpenAI-compatible server built in\n- Rich ecosystem (Ollama, LM Studio, llama-cpp-python)\n\n**Use alternatives instead:**\n- **vLLM** — NVIDIA GPUs, PagedAttention, Python-first, max throughput\n- **TensorRT-LLM** — Production NVIDIA (A100\u002FH100), maximum speed\n- **AWQ\u002FGPTQ** — Calibrated quantization for NVIDIA-only deployments\n- **bitsandbytes** — Simple HuggingFace transformers integration\n- **HQQ** — Fast calibration-free quantization\n\n## Quick start\n\n### Install\n\n```bash\n# macOS \u002F Linux (simplest)\nbrew install llama.cpp\n\n# Or build from source\ngit clone https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\ncd llama.cpp\nmake                        # CPU\nmake GGML_METAL=1           # Apple Silicon\nmake GGML_CUDA=1            # NVIDIA CUDA\nmake LLAMA_HIP=1            # AMD ROCm\n\n# Python bindings (optional)\npip install llama-cpp-python\n# With CUDA:   CMAKE_ARGS=\"-DGGML_CUDA=on\" pip install llama-cpp-python --force-reinstall --no-cache-dir\n# With Metal:  CMAKE_ARGS=\"-DGGML_METAL=on\" pip install llama-cpp-python --force-reinstall --no-cache-dir\n```\n\n### Download a pre-quantized GGUF\n\n```bash\n# TheBloke hosts most popular models pre-quantized\nhuggingface-cli download \\\n    TheBloke\u002FLlama-2-7B-Chat-GGUF \\\n    llama-2-7b-chat.Q4_K_M.gguf \\\n    --local-dir models\u002F\n```\n\n### Or convert a HuggingFace model to GGUF\n\n```bash\n# 1. Download HF model\nhuggingface-cli download meta-llama\u002FLlama-3.1-8B --local-dir .\u002Fllama-3.1-8b\n\n# 2. Convert to FP16 GGUF\npython convert_hf_to_gguf.py .\u002Fllama-3.1-8b \\\n    --outfile llama-3.1-8b-f16.gguf \\\n    --outtype f16\n\n# 3. Quantize to Q4_K_M\n.\u002Fllama-quantize llama-3.1-8b-f16.gguf llama-3.1-8b-q4_k_m.gguf Q4_K_M\n```\n\n### Run inference\n\n```bash\n# One-shot prompt\n.\u002Fllama-cli -m model.Q4_K_M.gguf -p \"Explain quantum computing\" -n 256\n\n# Interactive chat\n.\u002Fllama-cli -m model.Q4_K_M.gguf --interactive\n\n# With GPU offload\n.\u002Fllama-cli -m model.Q4_K_M.gguf -ngl 35 -p \"Hello!\"\n```\n\n### Serve an OpenAI-compatible API\n\n```bash\n.\u002Fllama-server \\\n    -m model.Q4_K_M.gguf \\\n    --host 0.0.0.0 \\\n    --port 8080 \\\n    -ngl 35 \\\n    -c 4096 \\\n    --parallel 4 \\\n    --cont-batching\n```\n\n```bash\ncurl http:\u002F\u002Flocalhost:8080\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"local\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}],\n    \"temperature\": 0.7,\n    \"max_tokens\": 100\n  }'\n```\n\n## Quantization formats (GGUF)\n\n### K-quant methods (recommended)\n\n| Type | Bits | Size (7B) | Quality | Use Case |\n|------|------|-----------|---------|----------|\n| Q2_K | 2.5 | ~2.8 GB | Low | Extreme compression (testing only) |\n| Q3_K_S | 3.0 | ~3.0 GB | Low-Med | Memory constrained |\n| Q3_K_M | 3.3 | ~3.3 GB | Medium | Fits small devices |\n| Q4_K_S | 4.0 | ~3.8 GB | Med-High | Speed critical |\n| **Q4_K_M** | 4.5 | ~4.1 GB | High | **Recommended default** |\n| Q5_K_S | 5.0 | ~4.6 GB | High | Quality focused |\n| Q5_K_M | 5.5 | ~4.8 GB | Very High | High quality |\n| Q6_K | 6.0 | ~5.5 GB | Excellent | Near-original |\n| Q8_0 | 8.0 | ~7.2 GB | Best | Maximum quality, minimal degradation |\n\n**Variant suffixes** — `_S` (Small, faster, lower quality), `_M` (Medium, balanced), `_L` (Large, better quality).\n\n**Legacy (Q4_0\u002FQ4_1\u002FQ5_0\u002FQ5_1) exist** but always prefer K-quants for better quality\u002Fsize ratio.\n\n**IQ quantization** — ultra-low-bit with importance-aware methods: IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_XS, IQ3_S, IQ4_XS. Require `--imatrix`.\n\n**Task-specific defaults:**\n- General chat \u002F assistants: Q4_K_M, or Q5_K_M if RAM allows\n- Code generation: Q5_K_M or Q6_K (higher precision helps)\n- Technical \u002F medical: Q6_K or Q8_0\n- Very large (70B, 405B) on consumer hardware: Q3_K_M or Q4_K_S\n- Raspberry Pi \u002F edge: Q2_K or Q3_K_S\n\n## Conversion workflows\n\n### Basic: HF → GGUF → quantized\n\n```bash\npython convert_hf_to_gguf.py .\u002Fmodel --outfile model-f16.gguf --outtype f16\n.\u002Fllama-quantize model-f16.gguf model-q4_k_m.gguf Q4_K_M\n.\u002Fllama-cli -m model-q4_k_m.gguf -p \"Hello!\" -n 50\n```\n\n### With importance matrix (imatrix) — better low-bit quality\n\n`imatrix` gives 10–20% perplexity improvement at Q4, essential at Q3 and below.\n\n```bash\n# 1. Convert to FP16 GGUF\npython convert_hf_to_gguf.py .\u002Fmodel --outfile model-f16.gguf\n\n# 2. Prepare calibration data (diverse text, ~100MB is ideal)\ncat > calibration.txt \u003C\u003C 'EOF'\nThe quick brown fox jumps over the lazy dog.\nMachine learning is a subset of artificial intelligence.\n# Add more diverse text samples...\nEOF\n\n# 3. Generate importance matrix\n.\u002Fllama-imatrix -m model-f16.gguf \\\n    -f calibration.txt \\\n    --chunk 512 \\\n    -o model.imatrix \\\n    -ngl 35\n\n# 4. Quantize with imatrix\n.\u002Fllama-quantize --imatrix model.imatrix \\\n    model-f16.gguf model-q4_k_m.gguf Q4_K_M\n```\n\n### Multi-quant batch\n\n```bash\n#!\u002Fbin\u002Fbash\nMODEL=\"llama-3.1-8b-f16.gguf\"\nIMATRIX=\"llama-3.1-8b.imatrix\"\n\n.\u002Fllama-imatrix -m $MODEL -f wiki.txt -o $IMATRIX -ngl 35\n\nfor QUANT in Q4_K_M Q5_K_M Q6_K Q8_0; do\n    OUTPUT=\"llama-3.1-8b-${QUANT,,}.gguf\"\n    .\u002Fllama-quantize --imatrix $IMATRIX $MODEL $OUTPUT $QUANT\n    echo \"Created: $OUTPUT ($(du -h $OUTPUT | cut -f1))\"\ndone\n```\n\n### Quality testing (perplexity)\n\n```bash\n.\u002Fllama-perplexity -m model.gguf -f wikitext-2-raw\u002Fwiki.test.raw -c 512\n# Baseline FP16: ~5.96  |  Q4_K_M: ~6.06 (+1.7%)  |  Q2_K: ~6.87 (+15.3%)\n```\n\n## Python bindings (llama-cpp-python)\n\n### Basic generation\n\n```python\nfrom llama_cpp import Llama\n\nllm = Llama(\n    model_path=\".\u002Fmodel-q4_k_m.gguf\",\n    n_ctx=4096,\n    n_gpu_layers=35,     # 0 for CPU only, 99 to offload everything\n    n_threads=8,\n)\n\noutput = llm(\n    \"What is machine learning?\",\n    max_tokens=256,\n    temperature=0.7,\n    stop=[\"\u003C\u002Fs>\", \"\\n\\n\"],\n)\nprint(output[\"choices\"][0][\"text\"])\n```\n\n### Chat completion + streaming\n\n```python\nllm = Llama(\n    model_path=\".\u002Fmodel-q4_k_m.gguf\",\n    n_ctx=4096,\n    n_gpu_layers=35,\n    chat_format=\"llama-3\",    # Or \"chatml\", \"mistral\", etc.\n)\n\n# Non-streaming\nresponse = llm.create_chat_completion(\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"What is Python?\"},\n    ],\n    max_tokens=256,\n    temperature=0.7,\n)\nprint(response[\"choices\"][0][\"message\"][\"content\"])\n\n# Streaming\nfor chunk in llm(\"Explain quantum computing:\", max_tokens=256, stream=True):\n    print(chunk[\"choices\"][0][\"text\"], end=\"\", flush=True)\n```\n\n### Embeddings\n\n```python\nllm = Llama(model_path=\".\u002Fmodel-q4_k_m.gguf\", embedding=True, n_gpu_layers=35)\nvec = llm.embed(\"This is a test sentence.\")\nprint(f\"Embedding dimension: {len(vec)}\")\n```\n\n## Hardware acceleration\n\n### Apple Silicon (Metal)\n\n```bash\nmake clean && make GGML_METAL=1\n.\u002Fllama-cli -m model.gguf -ngl 99 -p \"Hello\"   # offload all layers\n```\n\n```python\nllm = Llama(\n    model_path=\"model.gguf\",\n    n_gpu_layers=99,     # Offload everything\n    n_threads=1,         # Metal handles parallelism\n)\n```\n\nPerformance: M3 Max ~40–60 tok\u002Fs on Llama 2-7B Q4_K_M.\n\n### NVIDIA (CUDA)\n\n```bash\nmake clean && make GGML_CUDA=1\n.\u002Fllama-cli -m model.gguf -ngl 35 -p \"Hello\"\n\n# Hybrid for large models\n.\u002Fllama-cli -m llama-70b.Q4_K_M.gguf -ngl 20   # GPU: 20 layers, CPU: rest\n\n# Multi-GPU split\n.\u002Fllama-cli -m large-model.gguf --tensor-split 0.5,0.5 -ngl 60\n```\n\n### AMD (ROCm)\n\n```bash\nmake LLAMA_HIP=1\n.\u002Fllama-cli -m model.gguf -ngl 999\n```\n\n### CPU\n\n```bash\n# Match PHYSICAL cores, not logical\n.\u002Fllama-cli -m model.gguf -t 8 -p \"Hello\"\n\n# BLAS acceleration (2–3× speedup)\nmake LLAMA_OPENBLAS=1\n```\n\n```python\nllm = Llama(\n    model_path=\"model.gguf\",\n    n_gpu_layers=0,\n    n_threads=8,\n    n_batch=512,         # Larger batch = faster prompt processing\n)\n```\n\n## Performance benchmarks\n\n### CPU (Llama 2-7B Q4_K_M)\n\n| CPU | Threads | Speed |\n|-----|---------|-------|\n| Apple M3 Max (Metal) | 16 | 50 tok\u002Fs |\n| AMD Ryzen 9 7950X | 32 | 35 tok\u002Fs |\n| Intel i9-13900K | 32 | 30 tok\u002Fs |\n\n### GPU offloading on RTX 4090\n\n| Layers GPU | Speed | VRAM |\n|------------|-------|------|\n| 0 (CPU only) | 30 tok\u002Fs | 0 GB |\n| 20 (hybrid) | 80 tok\u002Fs | 8 GB |\n| 35 (all) | 120 tok\u002Fs | 12 GB |\n\n## Supported models\n\n- **LLaMA family**: Llama 2 (7B\u002F13B\u002F70B), Llama 3 (8B\u002F70B\u002F405B), Code Llama\n- **Mistral family**: Mistral 7B, Mixtral 8x7B\u002F8x22B\n- **Other**: Falcon, BLOOM, GPT-J, Phi-3, Gemma, Qwen, LLaVA (vision), Whisper (audio)\n\nFind GGUF models: https:\u002F\u002Fhuggingface.co\u002Fmodels?library=gguf\n\n## Ecosystem integrations\n\n### Ollama\n\n```bash\ncat > Modelfile \u003C\u003C 'EOF'\nFROM .\u002Fmodel-q4_k_m.gguf\nTEMPLATE \"\"\"{{ .System }}\n{{ .Prompt }}\"\"\"\nPARAMETER temperature 0.7\nPARAMETER num_ctx 4096\nEOF\n\nollama create mymodel -f Modelfile\nollama run mymodel \"Hello!\"\n```\n\n### LM Studio\n\n1. Place GGUF file in `~\u002F.cache\u002Flm-studio\u002Fmodels\u002F`\n2. Open LM Studio and select the model\n3. Configure context length and GPU offload, start inference\n\n### text-generation-webui\n\n```bash\ncp model-q4_k_m.gguf text-generation-webui\u002Fmodels\u002F\npython server.py --model model-q4_k_m.gguf --loader llama.cpp --n-gpu-layers 35\n```\n\n### OpenAI client → llama-server\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http:\u002F\u002Flocalhost:8080\u002Fv1\", api_key=\"not-needed\")\nresponse = client.chat.completions.create(\n    model=\"local-model\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n    max_tokens=256,\n)\nprint(response.choices[0].message.content)\n```\n\n## Best practices\n\n1. **Use K-quants** — Q4_K_M is the recommended default\n2. **Use imatrix** for Q4 and below (calibration improves quality substantially)\n3. **Offload as many layers as VRAM allows** — start high, reduce by 5 on OOM\n4. **Thread count** — match physical cores, not logical\n5. **Batch size** — increase `n_batch` (e.g. 512) for faster prompt processing\n6. **Context** — start at 4096, grow only as needed (memory scales with ctx)\n7. **Flash Attention** — add `--flash-attn` if your build supports it\n\n## Common issues (quick fixes)\n\n**Model loads slowly** — use `--mmap` for memory-mapped loading.\n\n**Out of memory (GPU)** — reduce `-ngl`, use a smaller quant (Q4_K_S \u002F Q3_K_M), or quantize the KV cache:\n```python\nLlama(model_path=\"...\", type_k=2, type_v=2, n_gpu_layers=35)  # Q4_0 KV cache\n```\n\n**Garbage output** — wrong `chat_format`, temperature too high, or model file corrupted. Test with `temperature=0.1` and verify FP16 baseline works.\n\n**Connection refused (server)** — bind to `--host 0.0.0.0`, check `lsof -i :8080`.\n\nSee `references\u002Ftroubleshooting.md` for the full playbook.\n\n## References\n\n- **[advanced-usage.md](references\u002Fadvanced-usage.md)** — speculative decoding, batched inference, grammar-constrained generation, LoRA, multi-GPU, custom builds, benchmark scripts\n- **[quantization.md](references\u002Fquantization.md)** — perplexity tables, use-case guide, model size scaling (7B\u002F13B\u002F70B RAM needs), imatrix deep dive\n- **[server.md](references\u002Fserver.md)** — OpenAI API endpoints, Docker deployment, NGINX load balancing, monitoring\n- **[optimization.md](references\u002Foptimization.md)** — CPU threading, BLAS, GPU offload heuristics, batch tuning, benchmarks\n- **[troubleshooting.md](references\u002Ftroubleshooting.md)** — install\u002Fconvert\u002Fquantize\u002Finference\u002Fserver issues, Apple Silicon, debugging\n\n## Resources\n\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\n- **Python bindings**: https:\u002F\u002Fgithub.com\u002Fabetlen\u002Fllama-cpp-python\n- **Pre-quantized models**: https:\u002F\u002Fhuggingface.co\u002FTheBloke\n- **GGUF converter Space**: https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fggml-org\u002Fgguf-my-repo\n- **License**: MIT\n",{"name":5,"author":48,"license":14,"version":49,"metadata":50,"description":7,"dependencies":60},"Orchestra Research","2.0.0",{"hermes":51},{"tags":52},[28,29,53,27,54,55,56,57,58,59,26],"Quantization","Apple Silicon","Edge Deployment","Non-NVIDIA","AMD GPUs","Intel GPUs","Embedded",[61],"llama-cpp-python>=0.2.0","main","17ee50c38aefa9426a89d7e8ee167fdc4c184fb7","skills\u002Fmlops\u002Finference\u002Fllama-cpp",0,"https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup","https:\u002F\u002Fgithub.com\u002Fxuanxuan1983","2026-05-24 00:14:48",{"source":8,"skillName":5,"commands":70,"prompt":80,"zipAvailable":81},[71,74,77],{"tool":72,"command":73},"npx","npx skills add xuanxuan1983\u002Fhermes-skills-backup --skill llama-cpp",{"tool":75,"command":76},"bunx","bunx skills add xuanxuan1983\u002Fhermes-skills-backup --skill llama-cpp",{"tool":78,"command":79},"pnpm","pnpm dlx skills add xuanxuan1983\u002Fhermes-skills-backup --skill llama-cpp","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Finference\u002Fllama-cpp\u002FSKILL.md",true,[83,89,93],{"path":84,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":87,"sourceHtmlUrl":88},"references\u002Foptimization.md","REFERENCE","参考文档",1659,"https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Finference\u002Fllama-cpp\u002Freferences\u002Foptimization.md",{"path":90,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":91,"sourceHtmlUrl":92},"references\u002Fquantization.md",4956,"https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Finference\u002Fllama-cpp\u002Freferences\u002Fquantization.md",{"path":94,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":95,"sourceHtmlUrl":96},"references\u002Fserver.md",2259,"https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Finference\u002Fllama-cpp\u002Freferences\u002Fserver.md",[98,139,167,183,205,226],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":118,"publishedAt":43},"01M71JFHVW1W4XE013TY9ZYN9P","writing-plans","Use when you have a spec or requirements for a multi-step task. Creates comprehensive implementation plans with bite-sized tasks, exact file paths, and complete code examples.","https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fsoftware-development\u002Fwriting-plans\u002FSKILL.md",[104,107,110,115],{"categoryNo":105,"categoryName":106,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":108,"categoryName":109,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":111,"categoryName":112,"taxonomyType":113,"taxonomyTypeDesc":114},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",{"categoryNo":116,"categoryName":117,"taxonomyType":113,"taxonomyTypeDesc":114},"01JBSEEDCAT000000000000036","内容创作者",[119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138],"writing","writing plans","plans","test-driven","test-driven-development","superpowers","design","when","workflow","you","code","tasks","subagent","code-review","review","documentation","paths","planning","development","for",{"skillNo":140,"name":123,"nameZh":6,"description":141,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":142,"categories":143,"keywords":152,"publishedAt":43},"01M71JFF859RV0XGWS206X1TMB","Use when implementing any feature or bugfix, before writing implementation code. Enforces RED-GREEN-REFACTOR cycle with test-first approach.","https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fsoftware-development\u002Ftest-driven-development\u002FSKILL.md",[144,147,150,151],{"categoryNo":145,"categoryName":146,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000007","代码审查",{"categoryNo":148,"categoryName":149,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000008","测试与质量",{"categoryNo":108,"categoryName":109,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":116,"categoryName":117,"taxonomyType":113,"taxonomyTypeDesc":114},[153,137,154,155,156,157,158,159,160,161,124,119,162,126,129,131,163,164,165,166],"test driven development","test","driven","test-first","green","cycle","red-green-refactor","testing","implementing","debugging","systematic","bugfix","refactor","systematic-debugging",{"skillNo":168,"name":166,"nameZh":6,"description":169,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":170,"categories":171,"keywords":173,"publishedAt":43},"01M71JFCA5S48PZ5M7DG31R8K5","Use when encountering any bug, test failure, or unexpected behavior. 4-phase root cause investigation — NO fixes without understanding the problem first.","https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fsoftware-development\u002Fsystematic-debugging\u002FSKILL.md",[172],{"categoryNo":148,"categoryName":149,"taxonomyType":20,"taxonomyTypeDesc":21},[174,162,163,122,175,176,123,124,119,126,131,177,178,137,179,180,121,181,182,154],"systematic debugging","problem","problem-solving","phase","bug","fixes","root cause","use","root-cause",{"skillNo":184,"name":185,"nameZh":6,"description":186,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":187,"categories":188,"keywords":198,"publishedAt":43},"01M71JF9RQ092KM5ZXCBBT6FS1","subagent-driven-development","Use when executing implementation plans with independent tasks. Dispatches fresh delegate_task per task with two-stage review (spec compliance then code quality).","https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fsoftware-development\u002Fsubagent-driven-development\u002FSKILL.md",[189,190,193,194,195],{"categoryNo":108,"categoryName":109,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":191,"categoryName":192,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":111,"categoryName":112,"taxonomyType":113,"taxonomyTypeDesc":114},{"categoryNo":116,"categoryName":117,"taxonomyType":113,"taxonomyTypeDesc":114},{"categoryNo":196,"categoryName":197,"taxonomyType":113,"taxonomyTypeDesc":114},"01JBSEEDCAT000000000000037","律师与法务",[199,131,137,155,122,123,200,201,124,119,126,127,129,130,132,133,202,203,204,121],"subagent driven development","delegate","delegation","fresh","parallel","code quality",{"skillNo":206,"name":207,"nameZh":6,"description":208,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":209,"categories":210,"keywords":213,"publishedAt":43},"01M71JF76DA8ARZ4JRYR4VBRWM","requesting-code-review","Pre-commit verification pipeline — static security scan, baseline-aware quality gates, independent reviewer subagent, and auto-fix loop. Use after code changes and before committing, pushing, or opening a PR.","https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fsoftware-development\u002Frequesting-code-review\u002FSKILL.md",[211,212],{"categoryNo":145,"categoryName":146,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":111,"categoryName":112,"taxonomyType":113,"taxonomyTypeDesc":114},[214,215,129,133,122,216,217,123,218,219,124,220,119,221,222,223,131,132,224,225],"code review","requesting code review","pipeline","static","fix","auto-fix","auto","loop","security scan","commit","reviewer","verification",{"skillNo":227,"name":228,"nameZh":6,"description":229,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":230,"categories":231,"keywords":232,"publishedAt":43},"01M71JF52HZT5VQXFVF9K2H76K","plan","Plan mode for Hermes — inspect context, write a markdown plan into the active workspace's `.hermes\u002Fplans\u002F` directory, and do not execute the work.","https:\u002F\u002Fgithub.com\u002Fxuanxuan1983\u002Fhermes-skills-backup\u002Fblob\u002Fmain\u002Fskills\u002Fsoftware-development\u002Fplan\u002FSKILL.md",[],[228,233,234,136,137,138,235,236,121,119,127,131,237,238,239,240,241,242,243,244],"plan-mode","do","inspect","mode","work","markdown","active","directory","workspace","plan mode","write","implementation"]