[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78DV57X65S0XT75T74RB7WD":3,"skill-same:01M78DV57X65S0XT75T74RB7WD":76},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":40,"commitSha":41,"skillDirectory":42,"forksCount":14,"repositoryHtmlUrl":43,"repositoryHomepageUrl":6,"ownerHtmlUrl":44,"ownerBlogUrl":45,"ownerDescription":46,"updatedAt":47,"install":48,"resources":61},"01M78DV57X65S0XT75T74RB7WD","guidance",null,"Control LLM output with regex and grammars, guarantee valid JSON\u002FXML\u002Fcode generation, enforce structured formats, and build multi-step workflows with Guidance - Microsoft Research's constrained generation framework","pvbang\u002Fnexus","pvbang","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F89725712?v=4","01M78B6R1DXZZBTN9G0XXBBE0A","Phan Văn Bằng","01M78B6R2T4DYQ9ZY4GJR0Q3V4",0,"MIT","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fguidance\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: guidance\ndescription: Control LLM output with regex and grammars, guarantee valid JSON\u002FXML\u002Fcode generation, enforce structured formats, and build multi-step workflows with Guidance - Microsoft Research's constrained generation framework\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ndependencies: [guidance, transformers]\nmetadata:\n  hermes:\n    tags: [Prompt Engineering, Guidance, Constrained Generation, Structured Output, JSON Validation, Grammar, Microsoft Research, Format Enforcement, Multi-Step Workflows]\n\n---\n\n# Guidance: Constrained LLM Generation\n\n## When to Use This Skill\n\nUse Guidance when you need to:\n- **Control LLM output syntax** with regex or grammars\n- **Guarantee valid JSON\u002FXML\u002Fcode** generation\n- **Reduce latency** vs traditional prompting approaches\n- **Enforce structured formats** (dates, emails, IDs, etc.)\n- **Build multi-step workflows** with Pythonic control flow\n- **Prevent invalid outputs** through grammatical constraints\n\n**GitHub Stars**: 18,000+ | **From**: Microsoft Research\n\n## Installation\n\n```bash\n# Base installation\npip install guidance\n\n# With specific backends\npip install guidance[transformers]  # Hugging Face models\npip install guidance[llama_cpp]     # llama.cpp models\n```\n\n## Quick Start\n\n### Basic Example: Structured Generation\n\n```python\nfrom guidance import models, gen\n\n# Load model (supports OpenAI, Transformers, llama.cpp)\nlm = models.OpenAI(\"gpt-4\")\n\n# Generate with constraints\nresult = lm + \"The capital of France is \" + gen(\"capital\", max_tokens=5)\n\nprint(result[\"capital\"])  # \"Paris\"\n```\n\n### With Anthropic Claude\n\n```python\nfrom guidance import models, gen, system, user, assistant\n\n# Configure Claude\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# Use context managers for chat format\nwith system():\n    lm += \"You are a helpful assistant.\"\n\nwith user():\n    lm += \"What is the capital of France?\"\n\nwith assistant():\n    lm += gen(max_tokens=20)\n```\n\n## Core Concepts\n\n### 1. Context Managers\n\nGuidance uses Pythonic context managers for chat-style interactions.\n\n```python\nfrom guidance import system, user, assistant, gen\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# System message\nwith system():\n    lm += \"You are a JSON generation expert.\"\n\n# User message\nwith user():\n    lm += \"Generate a person object with name and age.\"\n\n# Assistant response\nwith assistant():\n    lm += gen(\"response\", max_tokens=100)\n\nprint(lm[\"response\"])\n```\n\n**Benefits:**\n- Natural chat flow\n- Clear role separation\n- Easy to read and maintain\n\n### 2. Constrained Generation\n\nGuidance ensures outputs match specified patterns using regex or grammars.\n\n#### Regex Constraints\n\n```python\nfrom guidance import models, gen\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# Constrain to valid email format\nlm += \"Email: \" + gen(\"email\", regex=r\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}\")\n\n# Constrain to date format (YYYY-MM-DD)\nlm += \"Date: \" + gen(\"date\", regex=r\"\\d{4}-\\d{2}-\\d{2}\")\n\n# Constrain to phone number\nlm += \"Phone: \" + gen(\"phone\", regex=r\"\\d{3}-\\d{3}-\\d{4}\")\n\nprint(lm[\"email\"])  # Guaranteed valid email\nprint(lm[\"date\"])   # Guaranteed YYYY-MM-DD format\n```\n\n**How it works:**\n- Regex converted to grammar at token level\n- Invalid tokens filtered during generation\n- Model can only produce matching outputs\n\n#### Selection Constraints\n\n```python\nfrom guidance import models, gen, select\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# Constrain to specific choices\nlm += \"Sentiment: \" + select([\"positive\", \"negative\", \"neutral\"], name=\"sentiment\")\n\n# Multiple-choice selection\nlm += \"Best answer: \" + select(\n    [\"A) Paris\", \"B) London\", \"C) Berlin\", \"D) Madrid\"],\n    name=\"answer\"\n)\n\nprint(lm[\"sentiment\"])  # One of: positive, negative, neutral\nprint(lm[\"answer\"])     # One of: A, B, C, or D\n```\n\n### 3. Token Healing\n\nGuidance automatically \"heals\" token boundaries between prompt and generation.\n\n**Problem:** Tokenization creates unnatural boundaries.\n\n```python\n# Without token healing\nprompt = \"The capital of France is \"\n# Last token: \" is \"\n# First generated token might be \" Par\" (with leading space)\n# Result: \"The capital of France is  Paris\" (double space!)\n```\n\n**Solution:** Guidance backs up one token and regenerates.\n\n```python\nfrom guidance import models, gen\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# Token healing enabled by default\nlm += \"The capital of France is \" + gen(\"capital\", max_tokens=5)\n# Result: \"The capital of France is Paris\" (correct spacing)\n```\n\n**Benefits:**\n- Natural text boundaries\n- No awkward spacing issues\n- Better model performance (sees natural token sequences)\n\n### 4. Grammar-Based Generation\n\nDefine complex structures using context-free grammars.\n\n```python\nfrom guidance import models, gen\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# JSON grammar (simplified)\njson_grammar = \"\"\"\n{\n    \"name\": \u003Cgen name regex=\"[A-Za-z ]+\" max_tokens=20>,\n    \"age\": \u003Cgen age regex=\"[0-9]+\" max_tokens=3>,\n    \"email\": \u003Cgen email regex=\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\\\.[a-zA-Z]{2,}\" max_tokens=50>\n}\n\"\"\"\n\n# Generate valid JSON\nlm += gen(\"person\", grammar=json_grammar)\n\nprint(lm[\"person\"])  # Guaranteed valid JSON structure\n```\n\n**Use cases:**\n- Complex structured outputs\n- Nested data structures\n- Programming language syntax\n- Domain-specific languages\n\n### 5. Guidance Functions\n\nCreate reusable generation patterns with the `@guidance` decorator.\n\n```python\nfrom guidance import guidance, gen, models\n\n@guidance\ndef generate_person(lm):\n    \"\"\"Generate a person with name and age.\"\"\"\n    lm += \"Name: \" + gen(\"name\", max_tokens=20, stop=\"\\n\")\n    lm += \"\\nAge: \" + gen(\"age\", regex=r\"[0-9]+\", max_tokens=3)\n    return lm\n\n# Use the function\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\nlm = generate_person(lm)\n\nprint(lm[\"name\"])\nprint(lm[\"age\"])\n```\n\n**Stateful Functions:**\n\n```python\n@guidance(stateless=False)\ndef react_agent(lm, question, tools, max_rounds=5):\n    \"\"\"ReAct agent with tool use.\"\"\"\n    lm += f\"Question: {question}\\n\\n\"\n\n    for i in range(max_rounds):\n        # Thought\n        lm += f\"Thought {i+1}: \" + gen(\"thought\", stop=\"\\n\")\n\n        # Action\n        lm += \"\\nAction: \" + select(list(tools.keys()), name=\"action\")\n\n        # Execute tool\n        tool_result = tools[lm[\"action\"]]()\n        lm += f\"\\nObservation: {tool_result}\\n\\n\"\n\n        # Check if done\n        lm += \"Done? \" + select([\"Yes\", \"No\"], name=\"completed\")\n        if lm[\"completed\"] == \"Yes\":\n            break\n\n    # Final answer\n    lm += \"\\nFinal Answer: \" + gen(\"answer\", max_tokens=100)\n    return lm\n```\n\n## Backend Configuration\n\n### Anthropic Claude\n\n```python\nfrom guidance import models\n\nlm = models.Anthropic(\n    model=\"claude-sonnet-4-5-20250929\",\n    api_key=\"your-api-key\"  # Or set ANTHROPIC_API_KEY env var\n)\n```\n\n### OpenAI\n\n```python\nlm = models.OpenAI(\n    model=\"gpt-4o-mini\",\n    api_key=\"your-api-key\"  # Or set OPENAI_API_KEY env var\n)\n```\n\n### Local Models (Transformers)\n\n```python\nfrom guidance.models import Transformers\n\nlm = Transformers(\n    \"microsoft\u002FPhi-4-mini-instruct\",\n    device=\"cuda\"  # Or \"cpu\"\n)\n```\n\n### Local Models (llama.cpp)\n\n```python\nfrom guidance.models import LlamaCpp\n\nlm = LlamaCpp(\n    model_path=\"\u002Fpath\u002Fto\u002Fmodel.gguf\",\n    n_ctx=4096,\n    n_gpu_layers=35\n)\n```\n\n## Common Patterns\n\n### Pattern 1: JSON Generation\n\n```python\nfrom guidance import models, gen, system, user, assistant\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\nwith system():\n    lm += \"You generate valid JSON.\"\n\nwith user():\n    lm += \"Generate a user profile with name, age, and email.\"\n\nwith assistant():\n    lm += \"\"\"{\n    \"name\": \"\"\" + gen(\"name\", regex=r'\"[A-Za-z ]+\"', max_tokens=30) + \"\"\",\n    \"age\": \"\"\" + gen(\"age\", regex=r\"[0-9]+\", max_tokens=3) + \"\"\",\n    \"email\": \"\"\" + gen(\"email\", regex=r'\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}\"', max_tokens=50) + \"\"\"\n}\"\"\"\n\nprint(lm)  # Valid JSON guaranteed\n```\n\n### Pattern 2: Classification\n\n```python\nfrom guidance import models, gen, select\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\ntext = \"This product is amazing! I love it.\"\n\nlm += f\"Text: {text}\\n\"\nlm += \"Sentiment: \" + select([\"positive\", \"negative\", \"neutral\"], name=\"sentiment\")\nlm += \"\\nConfidence: \" + gen(\"confidence\", regex=r\"[0-9]+\", max_tokens=3) + \"%\"\n\nprint(f\"Sentiment: {lm['sentiment']}\")\nprint(f\"Confidence: {lm['confidence']}%\")\n```\n\n### Pattern 3: Multi-Step Reasoning\n\n```python\nfrom guidance import models, gen, guidance\n\n@guidance\ndef chain_of_thought(lm, question):\n    \"\"\"Generate answer with step-by-step reasoning.\"\"\"\n    lm += f\"Question: {question}\\n\\n\"\n\n    # Generate multiple reasoning steps\n    for i in range(3):\n        lm += f\"Step {i+1}: \" + gen(f\"step_{i+1}\", stop=\"\\n\", max_tokens=100) + \"\\n\"\n\n    # Final answer\n    lm += \"\\nTherefore, the answer is: \" + gen(\"answer\", max_tokens=50)\n\n    return lm\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\nlm = chain_of_thought(lm, \"What is 15% of 200?\")\n\nprint(lm[\"answer\"])\n```\n\n### Pattern 4: ReAct Agent\n\n```python\nfrom guidance import models, gen, select, guidance\n\n@guidance(stateless=False)\ndef react_agent(lm, question):\n    \"\"\"ReAct agent with tool use.\"\"\"\n    tools = {\n        \"calculator\": lambda expr: eval(expr),\n        \"search\": lambda query: f\"Search results for: {query}\",\n    }\n\n    lm += f\"Question: {question}\\n\\n\"\n\n    for round in range(5):\n        # Thought\n        lm += f\"Thought: \" + gen(\"thought\", stop=\"\\n\") + \"\\n\"\n\n        # Action selection\n        lm += \"Action: \" + select([\"calculator\", \"search\", \"answer\"], name=\"action\")\n\n        if lm[\"action\"] == \"answer\":\n            lm += \"\\nFinal Answer: \" + gen(\"answer\", max_tokens=100)\n            break\n\n        # Action input\n        lm += \"\\nAction Input: \" + gen(\"action_input\", stop=\"\\n\") + \"\\n\"\n\n        # Execute tool\n        if lm[\"action\"] in tools:\n            result = tools[lm[\"action\"]](lm[\"action_input\"])\n            lm += f\"Observation: {result}\\n\\n\"\n\n    return lm\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\nlm = react_agent(lm, \"What is 25 * 4 + 10?\")\nprint(lm[\"answer\"])\n```\n\n### Pattern 5: Data Extraction\n\n```python\nfrom guidance import models, gen, guidance\n\n@guidance\ndef extract_entities(lm, text):\n    \"\"\"Extract structured entities from text.\"\"\"\n    lm += f\"Text: {text}\\n\\n\"\n\n    # Extract person\n    lm += \"Person: \" + gen(\"person\", stop=\"\\n\", max_tokens=30) + \"\\n\"\n\n    # Extract organization\n    lm += \"Organization: \" + gen(\"organization\", stop=\"\\n\", max_tokens=30) + \"\\n\"\n\n    # Extract date\n    lm += \"Date: \" + gen(\"date\", regex=r\"\\d{4}-\\d{2}-\\d{2}\", max_tokens=10) + \"\\n\"\n\n    # Extract location\n    lm += \"Location: \" + gen(\"location\", stop=\"\\n\", max_tokens=30) + \"\\n\"\n\n    return lm\n\ntext = \"Tim Cook announced at Apple Park on 2024-09-15 in Cupertino.\"\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\nlm = extract_entities(lm, text)\n\nprint(f\"Person: {lm['person']}\")\nprint(f\"Organization: {lm['organization']}\")\nprint(f\"Date: {lm['date']}\")\nprint(f\"Location: {lm['location']}\")\n```\n\n## Best Practices\n\n### 1. Use Regex for Format Validation\n\n```python\n# ✅ Good: Regex ensures valid format\nlm += \"Email: \" + gen(\"email\", regex=r\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}\")\n\n# ❌ Bad: Free generation may produce invalid emails\nlm += \"Email: \" + gen(\"email\", max_tokens=50)\n```\n\n### 2. Use select() for Fixed Categories\n\n```python\n# ✅ Good: Guaranteed valid category\nlm += \"Status: \" + select([\"pending\", \"approved\", \"rejected\"], name=\"status\")\n\n# ❌ Bad: May generate typos or invalid values\nlm += \"Status: \" + gen(\"status\", max_tokens=20)\n```\n\n### 3. Leverage Token Healing\n\n```python\n# Token healing is enabled by default\n# No special action needed - just concatenate naturally\nlm += \"The capital is \" + gen(\"capital\")  # Automatic healing\n```\n\n### 4. Use stop Sequences\n\n```python\n# ✅ Good: Stop at newline for single-line outputs\nlm += \"Name: \" + gen(\"name\", stop=\"\\n\")\n\n# ❌ Bad: May generate multiple lines\nlm += \"Name: \" + gen(\"name\", max_tokens=50)\n```\n\n### 5. Create Reusable Functions\n\n```python\n# ✅ Good: Reusable pattern\n@guidance\ndef generate_person(lm):\n    lm += \"Name: \" + gen(\"name\", stop=\"\\n\")\n    lm += \"\\nAge: \" + gen(\"age\", regex=r\"[0-9]+\")\n    return lm\n\n# Use multiple times\nlm = generate_person(lm)\nlm += \"\\n\\n\"\nlm = generate_person(lm)\n```\n\n### 6. Balance Constraints\n\n```python\n# ✅ Good: Reasonable constraints\nlm += gen(\"name\", regex=r\"[A-Za-z ]+\", max_tokens=30)\n\n# ❌ Too strict: May fail or be very slow\nlm += gen(\"name\", regex=r\"^(John|Jane)$\", max_tokens=10)\n```\n\n## Comparison to Alternatives\n\n| Feature | Guidance | Instructor | Outlines | LMQL |\n|---------|----------|------------|----------|------|\n| Regex Constraints | ✅ Yes | ❌ No | ✅ Yes | ✅ Yes |\n| Grammar Support | ✅ CFG | ❌ No | ✅ CFG | ✅ CFG |\n| Pydantic Validation | ❌ No | ✅ Yes | ✅ Yes | ❌ No |\n| Token Healing | ✅ Yes | ❌ No | ✅ Yes | ❌ No |\n| Local Models | ✅ Yes | ⚠️ Limited | ✅ Yes | ✅ Yes |\n| API Models | ✅ Yes | ✅ Yes | ⚠️ Limited | ✅ Yes |\n| Pythonic Syntax | ✅ Yes | ✅ Yes | ✅ Yes | ❌ SQL-like |\n| Learning Curve | Low | Low | Medium | High |\n\n**When to choose Guidance:**\n- Need regex\u002Fgrammar constraints\n- Want token healing\n- Building complex workflows with control flow\n- Using local models (Transformers, llama.cpp)\n- Prefer Pythonic syntax\n\n**When to choose alternatives:**\n- Instructor: Need Pydantic validation with automatic retrying\n- Outlines: Need JSON schema validation\n- LMQL: Prefer declarative query syntax\n\n## Performance Characteristics\n\n**Latency Reduction:**\n- 30-50% faster than traditional prompting for constrained outputs\n- Token healing reduces unnecessary regeneration\n- Grammar constraints prevent invalid token generation\n\n**Memory Usage:**\n- Minimal overhead vs unconstrained generation\n- Grammar compilation cached after first use\n- Efficient token filtering at inference time\n\n**Token Efficiency:**\n- Prevents wasted tokens on invalid outputs\n- No need for retry loops\n- Direct path to valid outputs\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fguidance.readthedocs.io\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fguidance-ai\u002Fguidance (18k+ stars)\n- **Notebooks**: https:\u002F\u002Fgithub.com\u002Fguidance-ai\u002Fguidance\u002Ftree\u002Fmain\u002Fnotebooks\n- **Discord**: Community support available\n\n## See Also\n\n- `references\u002Fconstraints.md` - Comprehensive regex and grammar patterns\n- `references\u002Fbackends.md` - Backend-specific configuration\n- `references\u002Fexamples.md` - Production-ready examples\n\n\n","# Guidance: Constrained LLM Generation\n\n## When to Use This Skill\n\nUse Guidance when you need to:\n- **Control LLM output syntax** with regex or grammars\n- **Guarantee valid JSON\u002FXML\u002Fcode** generation\n- **Reduce latency** vs traditional prompting approaches\n- **Enforce structured formats** (dates, emails, IDs, etc.)\n- **Build multi-step workflows** with Pythonic control flow\n- **Prevent invalid outputs** through grammatical constraints\n\n**GitHub Stars**: 18,000+ | **From**: Microsoft Research\n\n## Installation\n\n```bash\n# Base installation\npip install guidance\n\n# With specific backends\npip install guidance[transformers]  # Hugging Face models\npip install guidance[llama_cpp]     # llama.cpp models\n```\n\n## Quick Start\n\n### Basic Example: Structured Generation\n\n```python\nfrom guidance import models, gen\n\n# Load model (supports OpenAI, Transformers, llama.cpp)\nlm = models.OpenAI(\"gpt-4\")\n\n# Generate with constraints\nresult = lm + \"The capital of France is \" + gen(\"capital\", max_tokens=5)\n\nprint(result[\"capital\"])  # \"Paris\"\n```\n\n### With Anthropic Claude\n\n```python\nfrom guidance import models, gen, system, user, assistant\n\n# Configure Claude\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# Use context managers for chat format\nwith system():\n    lm += \"You are a helpful assistant.\"\n\nwith user():\n    lm += \"What is the capital of France?\"\n\nwith assistant():\n    lm += gen(max_tokens=20)\n```\n\n## Core Concepts\n\n### 1. Context Managers\n\nGuidance uses Pythonic context managers for chat-style interactions.\n\n```python\nfrom guidance import system, user, assistant, gen\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# System message\nwith system():\n    lm += \"You are a JSON generation expert.\"\n\n# User message\nwith user():\n    lm += \"Generate a person object with name and age.\"\n\n# Assistant response\nwith assistant():\n    lm += gen(\"response\", max_tokens=100)\n\nprint(lm[\"response\"])\n```\n\n**Benefits:**\n- Natural chat flow\n- Clear role separation\n- Easy to read and maintain\n\n### 2. Constrained Generation\n\nGuidance ensures outputs match specified patterns using regex or grammars.\n\n#### Regex Constraints\n\n```python\nfrom guidance import models, gen\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# Constrain to valid email format\nlm += \"Email: \" + gen(\"email\", regex=r\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}\")\n\n# Constrain to date format (YYYY-MM-DD)\nlm += \"Date: \" + gen(\"date\", regex=r\"\\d{4}-\\d{2}-\\d{2}\")\n\n# Constrain to phone number\nlm += \"Phone: \" + gen(\"phone\", regex=r\"\\d{3}-\\d{3}-\\d{4}\")\n\nprint(lm[\"email\"])  # Guaranteed valid email\nprint(lm[\"date\"])   # Guaranteed YYYY-MM-DD format\n```\n\n**How it works:**\n- Regex converted to grammar at token level\n- Invalid tokens filtered during generation\n- Model can only produce matching outputs\n\n#### Selection Constraints\n\n```python\nfrom guidance import models, gen, select\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# Constrain to specific choices\nlm += \"Sentiment: \" + select([\"positive\", \"negative\", \"neutral\"], name=\"sentiment\")\n\n# Multiple-choice selection\nlm += \"Best answer: \" + select(\n    [\"A) Paris\", \"B) London\", \"C) Berlin\", \"D) Madrid\"],\n    name=\"answer\"\n)\n\nprint(lm[\"sentiment\"])  # One of: positive, negative, neutral\nprint(lm[\"answer\"])     # One of: A, B, C, or D\n```\n\n### 3. Token Healing\n\nGuidance automatically \"heals\" token boundaries between prompt and generation.\n\n**Problem:** Tokenization creates unnatural boundaries.\n\n```python\n# Without token healing\nprompt = \"The capital of France is \"\n# Last token: \" is \"\n# First generated token might be \" Par\" (with leading space)\n# Result: \"The capital of France is  Paris\" (double space!)\n```\n\n**Solution:** Guidance backs up one token and regenerates.\n\n```python\nfrom guidance import models, gen\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# Token healing enabled by default\nlm += \"The capital of France is \" + gen(\"capital\", max_tokens=5)\n# Result: \"The capital of France is Paris\" (correct spacing)\n```\n\n**Benefits:**\n- Natural text boundaries\n- No awkward spacing issues\n- Better model performance (sees natural token sequences)\n\n### 4. Grammar-Based Generation\n\nDefine complex structures using context-free grammars.\n\n```python\nfrom guidance import models, gen\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\n# JSON grammar (simplified)\njson_grammar = \"\"\"\n{\n    \"name\": \u003Cgen name regex=\"[A-Za-z ]+\" max_tokens=20>,\n    \"age\": \u003Cgen age regex=\"[0-9]+\" max_tokens=3>,\n    \"email\": \u003Cgen email regex=\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\\\.[a-zA-Z]{2,}\" max_tokens=50>\n}\n\"\"\"\n\n# Generate valid JSON\nlm += gen(\"person\", grammar=json_grammar)\n\nprint(lm[\"person\"])  # Guaranteed valid JSON structure\n```\n\n**Use cases:**\n- Complex structured outputs\n- Nested data structures\n- Programming language syntax\n- Domain-specific languages\n\n### 5. Guidance Functions\n\nCreate reusable generation patterns with the `@guidance` decorator.\n\n```python\nfrom guidance import guidance, gen, models\n\n@guidance\ndef generate_person(lm):\n    \"\"\"Generate a person with name and age.\"\"\"\n    lm += \"Name: \" + gen(\"name\", max_tokens=20, stop=\"\\n\")\n    lm += \"\\nAge: \" + gen(\"age\", regex=r\"[0-9]+\", max_tokens=3)\n    return lm\n\n# Use the function\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\nlm = generate_person(lm)\n\nprint(lm[\"name\"])\nprint(lm[\"age\"])\n```\n\n**Stateful Functions:**\n\n```python\n@guidance(stateless=False)\ndef react_agent(lm, question, tools, max_rounds=5):\n    \"\"\"ReAct agent with tool use.\"\"\"\n    lm += f\"Question: {question}\\n\\n\"\n\n    for i in range(max_rounds):\n        # Thought\n        lm += f\"Thought {i+1}: \" + gen(\"thought\", stop=\"\\n\")\n\n        # Action\n        lm += \"\\nAction: \" + select(list(tools.keys()), name=\"action\")\n\n        # Execute tool\n        tool_result = tools[lm[\"action\"]]()\n        lm += f\"\\nObservation: {tool_result}\\n\\n\"\n\n        # Check if done\n        lm += \"Done? \" + select([\"Yes\", \"No\"], name=\"completed\")\n        if lm[\"completed\"] == \"Yes\":\n            break\n\n    # Final answer\n    lm += \"\\nFinal Answer: \" + gen(\"answer\", max_tokens=100)\n    return lm\n```\n\n## Backend Configuration\n\n### Anthropic Claude\n\n```python\nfrom guidance import models\n\nlm = models.Anthropic(\n    model=\"claude-sonnet-4-5-20250929\",\n    api_key=\"your-api-key\"  # Or set ANTHROPIC_API_KEY env var\n)\n```\n\n### OpenAI\n\n```python\nlm = models.OpenAI(\n    model=\"gpt-4o-mini\",\n    api_key=\"your-api-key\"  # Or set OPENAI_API_KEY env var\n)\n```\n\n### Local Models (Transformers)\n\n```python\nfrom guidance.models import Transformers\n\nlm = Transformers(\n    \"microsoft\u002FPhi-4-mini-instruct\",\n    device=\"cuda\"  # Or \"cpu\"\n)\n```\n\n### Local Models (llama.cpp)\n\n```python\nfrom guidance.models import LlamaCpp\n\nlm = LlamaCpp(\n    model_path=\"\u002Fpath\u002Fto\u002Fmodel.gguf\",\n    n_ctx=4096,\n    n_gpu_layers=35\n)\n```\n\n## Common Patterns\n\n### Pattern 1: JSON Generation\n\n```python\nfrom guidance import models, gen, system, user, assistant\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\nwith system():\n    lm += \"You generate valid JSON.\"\n\nwith user():\n    lm += \"Generate a user profile with name, age, and email.\"\n\nwith assistant():\n    lm += \"\"\"{\n    \"name\": \"\"\" + gen(\"name\", regex=r'\"[A-Za-z ]+\"', max_tokens=30) + \"\"\",\n    \"age\": \"\"\" + gen(\"age\", regex=r\"[0-9]+\", max_tokens=3) + \"\"\",\n    \"email\": \"\"\" + gen(\"email\", regex=r'\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}\"', max_tokens=50) + \"\"\"\n}\"\"\"\n\nprint(lm)  # Valid JSON guaranteed\n```\n\n### Pattern 2: Classification\n\n```python\nfrom guidance import models, gen, select\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\n\ntext = \"This product is amazing! I love it.\"\n\nlm += f\"Text: {text}\\n\"\nlm += \"Sentiment: \" + select([\"positive\", \"negative\", \"neutral\"], name=\"sentiment\")\nlm += \"\\nConfidence: \" + gen(\"confidence\", regex=r\"[0-9]+\", max_tokens=3) + \"%\"\n\nprint(f\"Sentiment: {lm['sentiment']}\")\nprint(f\"Confidence: {lm['confidence']}%\")\n```\n\n### Pattern 3: Multi-Step Reasoning\n\n```python\nfrom guidance import models, gen, guidance\n\n@guidance\ndef chain_of_thought(lm, question):\n    \"\"\"Generate answer with step-by-step reasoning.\"\"\"\n    lm += f\"Question: {question}\\n\\n\"\n\n    # Generate multiple reasoning steps\n    for i in range(3):\n        lm += f\"Step {i+1}: \" + gen(f\"step_{i+1}\", stop=\"\\n\", max_tokens=100) + \"\\n\"\n\n    # Final answer\n    lm += \"\\nTherefore, the answer is: \" + gen(\"answer\", max_tokens=50)\n\n    return lm\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\nlm = chain_of_thought(lm, \"What is 15% of 200?\")\n\nprint(lm[\"answer\"])\n```\n\n### Pattern 4: ReAct Agent\n\n```python\nfrom guidance import models, gen, select, guidance\n\n@guidance(stateless=False)\ndef react_agent(lm, question):\n    \"\"\"ReAct agent with tool use.\"\"\"\n    tools = {\n        \"calculator\": lambda expr: eval(expr),\n        \"search\": lambda query: f\"Search results for: {query}\",\n    }\n\n    lm += f\"Question: {question}\\n\\n\"\n\n    for round in range(5):\n        # Thought\n        lm += f\"Thought: \" + gen(\"thought\", stop=\"\\n\") + \"\\n\"\n\n        # Action selection\n        lm += \"Action: \" + select([\"calculator\", \"search\", \"answer\"], name=\"action\")\n\n        if lm[\"action\"] == \"answer\":\n            lm += \"\\nFinal Answer: \" + gen(\"answer\", max_tokens=100)\n            break\n\n        # Action input\n        lm += \"\\nAction Input: \" + gen(\"action_input\", stop=\"\\n\") + \"\\n\"\n\n        # Execute tool\n        if lm[\"action\"] in tools:\n            result = tools[lm[\"action\"]](lm[\"action_input\"])\n            lm += f\"Observation: {result}\\n\\n\"\n\n    return lm\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\nlm = react_agent(lm, \"What is 25 * 4 + 10?\")\nprint(lm[\"answer\"])\n```\n\n### Pattern 5: Data Extraction\n\n```python\nfrom guidance import models, gen, guidance\n\n@guidance\ndef extract_entities(lm, text):\n    \"\"\"Extract structured entities from text.\"\"\"\n    lm += f\"Text: {text}\\n\\n\"\n\n    # Extract person\n    lm += \"Person: \" + gen(\"person\", stop=\"\\n\", max_tokens=30) + \"\\n\"\n\n    # Extract organization\n    lm += \"Organization: \" + gen(\"organization\", stop=\"\\n\", max_tokens=30) + \"\\n\"\n\n    # Extract date\n    lm += \"Date: \" + gen(\"date\", regex=r\"\\d{4}-\\d{2}-\\d{2}\", max_tokens=10) + \"\\n\"\n\n    # Extract location\n    lm += \"Location: \" + gen(\"location\", stop=\"\\n\", max_tokens=30) + \"\\n\"\n\n    return lm\n\ntext = \"Tim Cook announced at Apple Park on 2024-09-15 in Cupertino.\"\n\nlm = models.Anthropic(\"claude-sonnet-4-5-20250929\")\nlm = extract_entities(lm, text)\n\nprint(f\"Person: {lm['person']}\")\nprint(f\"Organization: {lm['organization']}\")\nprint(f\"Date: {lm['date']}\")\nprint(f\"Location: {lm['location']}\")\n```\n\n## Best Practices\n\n### 1. Use Regex for Format Validation\n\n```python\n# ✅ Good: Regex ensures valid format\nlm += \"Email: \" + gen(\"email\", regex=r\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}\")\n\n# ❌ Bad: Free generation may produce invalid emails\nlm += \"Email: \" + gen(\"email\", max_tokens=50)\n```\n\n### 2. Use select() for Fixed Categories\n\n```python\n# ✅ Good: Guaranteed valid category\nlm += \"Status: \" + select([\"pending\", \"approved\", \"rejected\"], name=\"status\")\n\n# ❌ Bad: May generate typos or invalid values\nlm += \"Status: \" + gen(\"status\", max_tokens=20)\n```\n\n### 3. Leverage Token Healing\n\n```python\n# Token healing is enabled by default\n# No special action needed - just concatenate naturally\nlm += \"The capital is \" + gen(\"capital\")  # Automatic healing\n```\n\n### 4. Use stop Sequences\n\n```python\n# ✅ Good: Stop at newline for single-line outputs\nlm += \"Name: \" + gen(\"name\", stop=\"\\n\")\n\n# ❌ Bad: May generate multiple lines\nlm += \"Name: \" + gen(\"name\", max_tokens=50)\n```\n\n### 5. Create Reusable Functions\n\n```python\n# ✅ Good: Reusable pattern\n@guidance\ndef generate_person(lm):\n    lm += \"Name: \" + gen(\"name\", stop=\"\\n\")\n    lm += \"\\nAge: \" + gen(\"age\", regex=r\"[0-9]+\")\n    return lm\n\n# Use multiple times\nlm = generate_person(lm)\nlm += \"\\n\\n\"\nlm = generate_person(lm)\n```\n\n### 6. Balance Constraints\n\n```python\n# ✅ Good: Reasonable constraints\nlm += gen(\"name\", regex=r\"[A-Za-z ]+\", max_tokens=30)\n\n# ❌ Too strict: May fail or be very slow\nlm += gen(\"name\", regex=r\"^(John|Jane)$\", max_tokens=10)\n```\n\n## Comparison to Alternatives\n\n| Feature | Guidance | Instructor | Outlines | LMQL |\n|---------|----------|------------|----------|------|\n| Regex Constraints | ✅ Yes | ❌ No | ✅ Yes | ✅ Yes |\n| Grammar Support | ✅ CFG | ❌ No | ✅ CFG | ✅ CFG |\n| Pydantic Validation | ❌ No | ✅ Yes | ✅ Yes | ❌ No |\n| Token Healing | ✅ Yes | ❌ No | ✅ Yes | ❌ No |\n| Local Models | ✅ Yes | ⚠️ Limited | ✅ Yes | ✅ Yes |\n| API Models | ✅ Yes | ✅ Yes | ⚠️ Limited | ✅ Yes |\n| Pythonic Syntax | ✅ Yes | ✅ Yes | ✅ Yes | ❌ SQL-like |\n| Learning Curve | Low | Low | Medium | High |\n\n**When to choose Guidance:**\n- Need regex\u002Fgrammar constraints\n- Want token healing\n- Building complex workflows with control flow\n- Using local models (Transformers, llama.cpp)\n- Prefer Pythonic syntax\n\n**When to choose alternatives:**\n- Instructor: Need Pydantic validation with automatic retrying\n- Outlines: Need JSON schema validation\n- LMQL: Prefer declarative query syntax\n\n## Performance Characteristics\n\n**Latency Reduction:**\n- 30-50% faster than traditional prompting for constrained outputs\n- Token healing reduces unnecessary regeneration\n- Grammar constraints prevent invalid token generation\n\n**Memory Usage:**\n- Minimal overhead vs unconstrained generation\n- Grammar compilation cached after first use\n- Efficient token filtering at inference time\n\n**Token Efficiency:**\n- Prevents wasted tokens on invalid outputs\n- No need for retry loops\n- Direct path to valid outputs\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fguidance.readthedocs.io\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fguidance-ai\u002Fguidance (18k+ stars)\n- **Notebooks**: https:\u002F\u002Fgithub.com\u002Fguidance-ai\u002Fguidance\u002Ftree\u002Fmain\u002Fnotebooks\n- **Discord**: Community support available\n\n## See Also\n\n- `references\u002Fconstraints.md` - Comprehensive regex and grammar patterns\n- `references\u002Fbackends.md` - Backend-specific configuration\n- `references\u002Fexamples.md` - Production-ready examples\n\n\n",{"name":5,"author":24,"license":15,"version":25,"metadata":26,"description":7,"dependencies":38},"Orchestra Research","1.0.0",{"hermes":27},{"tags":28},[29,30,31,32,33,34,35,36,37],"Prompt Engineering","Guidance","Constrained Generation","Structured Output","JSON Validation","Grammar","Microsoft Research","Format Enforcement","Multi-Step Workflows",[5,39],"transformers","main","a0559bb2eb5ae47d603efe32afe4589f77bf4ded","src\u002Fopen-source\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fguidance","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus","https:\u002F\u002Fgithub.com\u002Fpvbang","pvbang.github.io\u002Fcv","\u003C\u002F> It's a pleasure to meet you","2026-11-13 16:52:49",{"source":8,"skillName":5,"commands":49,"prompt":59,"zipAvailable":60},[50,53,56],{"tool":51,"command":52},"npx","npx skills add pvbang\u002Fnexus --skill guidance",{"tool":54,"command":55},"bunx","bunx skills add pvbang\u002Fnexus --skill guidance",{"tool":57,"command":58},"pnpm","pnpm dlx skills add pvbang\u002Fnexus --skill guidance","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fguidance\u002FSKILL.md",true,[62,68,72],{"path":63,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":66,"sourceHtmlUrl":67},"references\u002Fbackends.md","REFERENCE","参考文档",12010,"https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fguidance\u002Freferences\u002Fbackends.md",{"path":69,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":70,"sourceHtmlUrl":71},"references\u002Fconstraints.md",15886,"https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fguidance\u002Freferences\u002Fconstraints.md",{"path":73,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":74,"sourceHtmlUrl":75},"references\u002Fexamples.md",20089,"https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Foptional-skills\u002Fmlops\u002Fguidance\u002Freferences\u002Fexamples.md",[77,86,94,101,108,115],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":81,"licenseFileUrl":82,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":19},"01M78EFV6FYHV11YKAKYXPND85","skill-creator","Create, edit, improve, or audit AgentSkills. Use when creating a new skill from scratch or when asked to improve, review, audit, tidy up, or clean up an existing skill or SKILL.md file. Also use when editing or restructuring a skill directory (moving files to references\u002F or scripts\u002F, removing stale content, validating against the AgentSkills spec). Triggers on phrases like \"create a skill\", \"author a skill\", \"tidy up a skill\", \"improve this skill\", \"review the skill\", \"clean up the skill\", \"audit the skill\".","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fopenclaw\u002Fskills\u002Fskill-creator\u002Flicense.txt","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fopenclaw\u002Fskills\u002Fskill-creator\u002FSKILL.md",[],[],{"skillNo":87,"name":88,"nameZh":6,"description":89,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":90,"sourceHtmlUrl":91,"categories":92,"keywords":93,"publishedAt":19},"01M78EC37DFKY5X6RFQ6XPE9EY","prose","OpenProse VM skill pack. Activate on any `prose` command, .prose files, or OpenProse mentions; orchestrates multi-agent workflows.","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fopenclaw\u002Fextensions\u002Fopen-prose\u002Fskills\u002Fprose\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fopenclaw\u002Fextensions\u002Fopen-prose\u002Fskills\u002Fprose\u002FSKILL.md",[],[],{"skillNo":95,"name":96,"nameZh":6,"description":97,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":98,"categories":99,"keywords":100,"publishedAt":19},"01M78E9C2RKT5BJ8ZR9KZTNS2A","writing-plans","Write implementation plans: bite-sized tasks, paths, code.","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Fskills\u002Fsoftware-development\u002Fwriting-plans\u002FSKILL.md",[],[],{"skillNo":102,"name":103,"nameZh":6,"description":104,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":105,"categories":106,"keywords":107,"publishedAt":19},"01M78E98QY8PH80R7QN1P8NMTH","test-driven-development","TDD: enforce RED-GREEN-REFACTOR, tests before code.","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Fskills\u002Fsoftware-development\u002Ftest-driven-development\u002FSKILL.md",[],[],{"skillNo":109,"name":110,"nameZh":6,"description":111,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":112,"categories":113,"keywords":114,"publishedAt":19},"01M78E93DGFNKP1WZ9WJ9S9N7H","systematic-debugging","4-phase root cause debugging: understand bugs before fixing.","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Fskills\u002Fsoftware-development\u002Fsystematic-debugging\u002FSKILL.md",[],[],{"skillNo":116,"name":117,"nameZh":6,"description":118,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":119,"categories":120,"keywords":121,"publishedAt":19},"01M78E8ZM4CER22PJT0WJ1RWMJ","subagent-driven-development","Execute plans via delegate_task subagents (2-stage review).","https:\u002F\u002Fgithub.com\u002Fpvbang\u002Fnexus\u002Fblob\u002Fmain\u002Fsrc\u002Fopen-source\u002Fhermes-agent\u002Fskills\u002Fsoftware-development\u002Fsubagent-driven-development\u002FSKILL.md",[],[]]