[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6DRDW1HC50YK7MWB93KADE4":3,"skill-same:01M6DRDW1HC50YK7MWB93KADE4":61},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":28,"linkOnly":29,"markdownContent":30,"markdownBody":31,"frontMatter":32,"refName":34,"commitSha":35,"skillDirectory":36,"forksCount":37,"repositoryHtmlUrl":38,"repositoryHomepageUrl":6,"ownerHtmlUrl":39,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":40,"install":41,"resources":54},"01M6DRDW1HC50YK7MWB93KADE4","regex-vs-llm-structured-text",null,"Decision framework for choosing between regex and LLM when parsing structured text — start with regex, add LLM only for low-confidence edge cases.","AntifragileTech\u002Fantifragile-claude-code","AntifragileTech","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F174357194?v=4","01M6DQCYMK4CDSQQAN0ZC72NTV","01M6DQCYS9YPW1WKMQ5TJMPQ43",9,"MIT","https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code\u002Fblob\u002Fmaster\u002Fassets\u002Fskills\u002Fdev\u002Fregex-vs-llm-structured-text\u002FSKILL.md",[],[18,19,20,21,22,23,24,25,26,27],"text","structured","regex","regex vs llm structured text","framework","edge","start","parsing","decision","only","2026-11-10 10:12:30",false,"---\nname: regex-vs-llm-structured-text\ndescription: Decision framework for choosing between regex and LLM when parsing structured text — start with regex, add LLM only for low-confidence edge cases.\norigin: ECC\n---\n\n# Regex vs LLM for Structured Text Parsing\n\nA practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.\n\n## When to Activate\n\n- Parsing structured text with repeating patterns (questions, forms, tables)\n- Deciding between regex and LLM for text extraction\n- Building hybrid pipelines that combine both approaches\n- Optimizing cost\u002Faccuracy tradeoffs in text processing\n\n## Decision Framework\n\n```\nIs the text format consistent and repeating?\n├── Yes (>90% follows a pattern) → Start with Regex\n│   ├── Regex handles 95%+ → Done, no LLM needed\n│   └── Regex handles \u003C95% → Add LLM for edge cases only\n└── No (free-form, highly variable) → Use LLM directly\n```\n\n## Architecture Pattern\n\n```\nSource Text\n    │\n    ▼\n[Regex Parser] ─── Extracts structure (95-98% accuracy)\n    │\n    ▼\n[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)\n    │\n    ▼\n[Confidence Scorer] ─── Flags low-confidence extractions\n    │\n    ├── High confidence (≥0.95) → Direct output\n    │\n    └── Low confidence (\u003C0.95) → [LLM Validator] → Output\n```\n\n## Implementation\n\n### 1. Regex Parser (Handles the Majority)\n\n```python\nimport re\nfrom dataclasses import dataclass\n\n@dataclass(frozen=True)\nclass ParsedItem:\n    id: str\n    text: str\n    choices: tuple[str, ...]\n    answer: str\n    confidence: float = 1.0\n\ndef parse_structured_text(content: str) -> list[ParsedItem]:\n    \"\"\"Parse structured text using regex patterns.\"\"\"\n    pattern = re.compile(\n        r\"(?P\u003Cid>\\d+)\\.\\s*(?P\u003Ctext>.+?)\\n\"\n        r\"(?P\u003Cchoices>(?:[A-D]\\..+?\\n)+)\"\n        r\"Answer:\\s*(?P\u003Canswer>[A-D])\",\n        re.MULTILINE | re.DOTALL,\n    )\n    items = []\n    for match in pattern.finditer(content):\n        choices = tuple(\n            c.strip() for c in re.findall(r\"[A-D]\\.\\s*(.+)\", match.group(\"choices\"))\n        )\n        items.append(ParsedItem(\n            id=match.group(\"id\"),\n            text=match.group(\"text\").strip(),\n            choices=choices,\n            answer=match.group(\"answer\"),\n        ))\n    return items\n```\n\n### 2. Confidence Scoring\n\nFlag items that may need LLM review:\n\n```python\n@dataclass(frozen=True)\nclass ConfidenceFlag:\n    item_id: str\n    score: float\n    reasons: tuple[str, ...]\n\ndef score_confidence(item: ParsedItem) -> ConfidenceFlag:\n    \"\"\"Score extraction confidence and flag issues.\"\"\"\n    reasons = []\n    score = 1.0\n\n    if len(item.choices) \u003C 3:\n        reasons.append(\"few_choices\")\n        score -= 0.3\n\n    if not item.answer:\n        reasons.append(\"missing_answer\")\n        score -= 0.5\n\n    if len(item.text) \u003C 10:\n        reasons.append(\"short_text\")\n        score -= 0.2\n\n    return ConfidenceFlag(\n        item_id=item.id,\n        score=max(0.0, score),\n        reasons=tuple(reasons),\n    )\n\ndef identify_low_confidence(\n    items: list[ParsedItem],\n    threshold: float = 0.95,\n) -> list[ConfidenceFlag]:\n    \"\"\"Return items below confidence threshold.\"\"\"\n    flags = [score_confidence(item) for item in items]\n    return [f for f in flags if f.score \u003C threshold]\n```\n\n### 3. LLM Validator (Edge Cases Only)\n\n```python\ndef validate_with_llm(\n    item: ParsedItem,\n    original_text: str,\n    client,\n) -> ParsedItem:\n    \"\"\"Use LLM to fix low-confidence extractions.\"\"\"\n    response = client.messages.create(\n        model=\"claude-haiku-4-5-20251001\",  # Cheapest model for validation\n        max_tokens=500,\n        messages=[{\n            \"role\": \"user\",\n            \"content\": (\n                f\"Extract the question, choices, and answer from this text.\\n\\n\"\n                f\"Text: {original_text}\\n\\n\"\n                f\"Current extraction: {item}\\n\\n\"\n                f\"Return corrected JSON if needed, or 'CORRECT' if accurate.\"\n            ),\n        }],\n    )\n    # Parse LLM response and return corrected item...\n    return corrected_item\n```\n\n### 4. Hybrid Pipeline\n\n```python\ndef process_document(\n    content: str,\n    *,\n    llm_client=None,\n    confidence_threshold: float = 0.95,\n) -> list[ParsedItem]:\n    \"\"\"Full pipeline: regex -> confidence check -> LLM for edge cases.\"\"\"\n    # Step 1: Regex extraction (handles 95-98%)\n    items = parse_structured_text(content)\n\n    # Step 2: Confidence scoring\n    low_confidence = identify_low_confidence(items, confidence_threshold)\n\n    if not low_confidence or llm_client is None:\n        return items\n\n    # Step 3: LLM validation (only for flagged items)\n    low_conf_ids = {f.item_id for f in low_confidence}\n    result = []\n    for item in items:\n        if item.id in low_conf_ids:\n            result.append(validate_with_llm(item, content, llm_client))\n        else:\n            result.append(item)\n\n    return result\n```\n\n## Real-World Metrics\n\nFrom a production quiz parsing pipeline (410 items):\n\n| Metric | Value |\n|--------|-------|\n| Regex success rate | 98.0% |\n| Low confidence items | 8 (2.0%) |\n| LLM calls needed | ~5 |\n| Cost savings vs all-LLM | ~95% |\n| Test coverage | 93% |\n\n## Best Practices\n\n- **Start with regex** — even imperfect regex gives you a baseline to improve\n- **Use confidence scoring** to programmatically identify what needs LLM help\n- **Use the cheapest LLM** for validation (Haiku-class models are sufficient)\n- **Never mutate** parsed items — return new instances from cleaning\u002Fvalidation steps\n- **TDD works well** for parsers — write tests for known patterns first, then edge cases\n- **Log metrics** (regex success rate, LLM call count) to track pipeline health\n\n## Anti-Patterns to Avoid\n\n- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)\n- Using regex for free-form, highly variable text (LLM is better here)\n- Skipping confidence scoring and hoping regex \"just works\"\n- Mutating parsed objects during cleaning\u002Fvalidation steps\n- Not testing edge cases (malformed input, missing fields, encoding issues)\n\n## When to Use\n\n- Quiz\u002Fexam question parsing\n- Form data extraction\n- Invoice\u002Freceipt processing\n- Document structure parsing (headers, sections, tables)\n- Any structured text with repeating patterns where cost matters\n","# Regex vs LLM for Structured Text Parsing\n\nA practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.\n\n## When to Activate\n\n- Parsing structured text with repeating patterns (questions, forms, tables)\n- Deciding between regex and LLM for text extraction\n- Building hybrid pipelines that combine both approaches\n- Optimizing cost\u002Faccuracy tradeoffs in text processing\n\n## Decision Framework\n\n```\nIs the text format consistent and repeating?\n├── Yes (>90% follows a pattern) → Start with Regex\n│   ├── Regex handles 95%+ → Done, no LLM needed\n│   └── Regex handles \u003C95% → Add LLM for edge cases only\n└── No (free-form, highly variable) → Use LLM directly\n```\n\n## Architecture Pattern\n\n```\nSource Text\n    │\n    ▼\n[Regex Parser] ─── Extracts structure (95-98% accuracy)\n    │\n    ▼\n[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)\n    │\n    ▼\n[Confidence Scorer] ─── Flags low-confidence extractions\n    │\n    ├── High confidence (≥0.95) → Direct output\n    │\n    └── Low confidence (\u003C0.95) → [LLM Validator] → Output\n```\n\n## Implementation\n\n### 1. Regex Parser (Handles the Majority)\n\n```python\nimport re\nfrom dataclasses import dataclass\n\n@dataclass(frozen=True)\nclass ParsedItem:\n    id: str\n    text: str\n    choices: tuple[str, ...]\n    answer: str\n    confidence: float = 1.0\n\ndef parse_structured_text(content: str) -> list[ParsedItem]:\n    \"\"\"Parse structured text using regex patterns.\"\"\"\n    pattern = re.compile(\n        r\"(?P\u003Cid>\\d+)\\.\\s*(?P\u003Ctext>.+?)\\n\"\n        r\"(?P\u003Cchoices>(?:[A-D]\\..+?\\n)+)\"\n        r\"Answer:\\s*(?P\u003Canswer>[A-D])\",\n        re.MULTILINE | re.DOTALL,\n    )\n    items = []\n    for match in pattern.finditer(content):\n        choices = tuple(\n            c.strip() for c in re.findall(r\"[A-D]\\.\\s*(.+)\", match.group(\"choices\"))\n        )\n        items.append(ParsedItem(\n            id=match.group(\"id\"),\n            text=match.group(\"text\").strip(),\n            choices=choices,\n            answer=match.group(\"answer\"),\n        ))\n    return items\n```\n\n### 2. Confidence Scoring\n\nFlag items that may need LLM review:\n\n```python\n@dataclass(frozen=True)\nclass ConfidenceFlag:\n    item_id: str\n    score: float\n    reasons: tuple[str, ...]\n\ndef score_confidence(item: ParsedItem) -> ConfidenceFlag:\n    \"\"\"Score extraction confidence and flag issues.\"\"\"\n    reasons = []\n    score = 1.0\n\n    if len(item.choices) \u003C 3:\n        reasons.append(\"few_choices\")\n        score -= 0.3\n\n    if not item.answer:\n        reasons.append(\"missing_answer\")\n        score -= 0.5\n\n    if len(item.text) \u003C 10:\n        reasons.append(\"short_text\")\n        score -= 0.2\n\n    return ConfidenceFlag(\n        item_id=item.id,\n        score=max(0.0, score),\n        reasons=tuple(reasons),\n    )\n\ndef identify_low_confidence(\n    items: list[ParsedItem],\n    threshold: float = 0.95,\n) -> list[ConfidenceFlag]:\n    \"\"\"Return items below confidence threshold.\"\"\"\n    flags = [score_confidence(item) for item in items]\n    return [f for f in flags if f.score \u003C threshold]\n```\n\n### 3. LLM Validator (Edge Cases Only)\n\n```python\ndef validate_with_llm(\n    item: ParsedItem,\n    original_text: str,\n    client,\n) -> ParsedItem:\n    \"\"\"Use LLM to fix low-confidence extractions.\"\"\"\n    response = client.messages.create(\n        model=\"claude-haiku-4-5-20251001\",  # Cheapest model for validation\n        max_tokens=500,\n        messages=[{\n            \"role\": \"user\",\n            \"content\": (\n                f\"Extract the question, choices, and answer from this text.\\n\\n\"\n                f\"Text: {original_text}\\n\\n\"\n                f\"Current extraction: {item}\\n\\n\"\n                f\"Return corrected JSON if needed, or 'CORRECT' if accurate.\"\n            ),\n        }],\n    )\n    # Parse LLM response and return corrected item...\n    return corrected_item\n```\n\n### 4. Hybrid Pipeline\n\n```python\ndef process_document(\n    content: str,\n    *,\n    llm_client=None,\n    confidence_threshold: float = 0.95,\n) -> list[ParsedItem]:\n    \"\"\"Full pipeline: regex -> confidence check -> LLM for edge cases.\"\"\"\n    # Step 1: Regex extraction (handles 95-98%)\n    items = parse_structured_text(content)\n\n    # Step 2: Confidence scoring\n    low_confidence = identify_low_confidence(items, confidence_threshold)\n\n    if not low_confidence or llm_client is None:\n        return items\n\n    # Step 3: LLM validation (only for flagged items)\n    low_conf_ids = {f.item_id for f in low_confidence}\n    result = []\n    for item in items:\n        if item.id in low_conf_ids:\n            result.append(validate_with_llm(item, content, llm_client))\n        else:\n            result.append(item)\n\n    return result\n```\n\n## Real-World Metrics\n\nFrom a production quiz parsing pipeline (410 items):\n\n| Metric | Value |\n|--------|-------|\n| Regex success rate | 98.0% |\n| Low confidence items | 8 (2.0%) |\n| LLM calls needed | ~5 |\n| Cost savings vs all-LLM | ~95% |\n| Test coverage | 93% |\n\n## Best Practices\n\n- **Start with regex** — even imperfect regex gives you a baseline to improve\n- **Use confidence scoring** to programmatically identify what needs LLM help\n- **Use the cheapest LLM** for validation (Haiku-class models are sufficient)\n- **Never mutate** parsed items — return new instances from cleaning\u002Fvalidation steps\n- **TDD works well** for parsers — write tests for known patterns first, then edge cases\n- **Log metrics** (regex success rate, LLM call count) to track pipeline health\n\n## Anti-Patterns to Avoid\n\n- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)\n- Using regex for free-form, highly variable text (LLM is better here)\n- Skipping confidence scoring and hoping regex \"just works\"\n- Mutating parsed objects during cleaning\u002Fvalidation steps\n- Not testing edge cases (malformed input, missing fields, encoding issues)\n\n## When to Use\n\n- Quiz\u002Fexam question parsing\n- Form data extraction\n- Invoice\u002Freceipt processing\n- Document structure parsing (headers, sections, tables)\n- Any structured text with repeating patterns where cost matters\n",{"name":5,"origin":33,"description":7},"ECC","master","c73386b305dec167db7654e0e6feacb462d4f9dc","assets\u002Fskills\u002Fdev\u002Fregex-vs-llm-structured-text",1,"https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code","https:\u002F\u002Fgithub.com\u002FAntifragileTech","2026-10-02 14:22:14",{"source":8,"skillName":5,"commands":42,"prompt":52,"zipAvailable":53},[43,46,49],{"tool":44,"command":45},"npx","npx skills add AntifragileTech\u002Fantifragile-claude-code --skill regex-vs-llm-structured-text",{"tool":47,"command":48},"bunx","bunx skills add AntifragileTech\u002Fantifragile-claude-code --skill regex-vs-llm-structured-text",{"tool":50,"command":51},"pnpm","pnpm dlx skills add AntifragileTech\u002Fantifragile-claude-code --skill regex-vs-llm-structured-text","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code\u002Fblob\u002Fmaster\u002Fassets\u002Fskills\u002Fdev\u002Fregex-vs-llm-structured-text\u002FSKILL.md",true,[55],{"path":56,"resourceType":57,"resourceTypeDesc":58,"sizeBytes":59,"sourceHtmlUrl":60},"index.json","REFERENCE","参考文档",627,"https:\u002F\u002Fgithub.com\u002FDCx7C5\u002Fai-marketplace\u002Fblob\u002Fmain\u002Fskills\u002Fintel\u002Fioc\u002Fsplunk\u002Fanalyze\u002Findex.json",[62,88,103,118,131,144],{"skillNo":63,"name":64,"nameZh":6,"description":65,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":81,"publishedAt":28},"01M6DT9Q6VYA615SCXR757TZZ7","writing-skills","Use when creating new skills, editing existing skills, or verifying skills work before deployment","https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code\u002Fblob\u002Fmaster\u002Fassets\u002Fskills\u002Fthinking\u002Fwriting-skills\u002FSKILL.md",[68,73,76],{"categoryNo":69,"categoryName":70,"taxonomyType":71,"taxonomyTypeDesc":72},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",{"categoryNo":74,"categoryName":75,"taxonomyType":71,"taxonomyTypeDesc":72},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":77,"categoryName":78,"taxonomyType":79,"taxonomyTypeDesc":80},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[82,83,84,85,86,87],"writing skills","writing","use","editing","work","deployment",{"skillNo":89,"name":90,"nameZh":6,"description":91,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":96,"publishedAt":28},"01M6DT9GFHDNY90P41WGAE1TV0","writing-plans","Use when you have a spec or requirements for a multi-step task, before touching code","https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code\u002Fblob\u002Fmaster\u002Fassets\u002Fskills\u002Fthinking\u002Fwriting-plans\u002FSKILL.md",[94,95],{"categoryNo":74,"categoryName":75,"taxonomyType":71,"taxonomyTypeDesc":72},{"categoryNo":77,"categoryName":78,"taxonomyType":79,"taxonomyTypeDesc":80},[97,98,83,99,100,84,101,102],"plans","writing plans","requirements","task","code","spec",{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":111,"publishedAt":28},"01M6DT9C6PFGD86WS3HY7YE3XG","docs:write-concisely","Apply writing rules to any documentation that humans will read. Makes your writing clearer, stronger, and more professional.","https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code\u002Fblob\u002Fmaster\u002Fassets\u002Fskills\u002Fthinking\u002Fwrite-concisely\u002FSKILL.md",[109,110],{"categoryNo":74,"categoryName":75,"taxonomyType":71,"taxonomyTypeDesc":72},{"categoryNo":77,"categoryName":78,"taxonomyType":79,"taxonomyTypeDesc":80},[112,113,114,115,116,117,83],"docs:write concisely","write","docs","read","Rules","documentation",{"skillNo":119,"name":120,"nameZh":6,"description":121,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":122,"categories":123,"keywords":124,"publishedAt":28},"01M6DT98AWQ27P0N1CG97WAG66","kaizen:why","Iterative Five Whys root cause analysis drilling from symptoms to fundamentals","https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code\u002Fblob\u002Fmaster\u002Fassets\u002Fskills\u002Fthinking\u002Fwhy\u002FSKILL.md",[],[120,125,126,127,128,129,130],"why","kaizen","issue","five whys","analysis","root cause analysis",{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":135,"categories":136,"keywords":137,"publishedAt":28},"01M6DT93EH25NAX19R3WGE1P53","verification-loop","A comprehensive verification system for Claude Code sessions.","https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code\u002Fblob\u002Fmaster\u002Fassets\u002Fskills\u002Fthinking\u002Fverification-loop\u002FSKILL.md",[],[138,139,140,141,101,142,143],"verification loop","verification","loop","sessions","claude code sessions","system",{"skillNo":145,"name":146,"nameZh":6,"description":147,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":148,"categories":149,"keywords":150,"publishedAt":28},"01M6DT90GW7Z7VCN98P9DVYSNC","verification-before-completion","Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always","https:\u002F\u002Fgithub.com\u002FAntifragileTech\u002Fantifragile-claude-code\u002Fblob\u002Fmaster\u002Fassets\u002Fskills\u002Fthinking\u002Fverification-before-completion\u002FSKILL.md",[],[151,139,152,86,84,153,154,155],"verification before completion","commands","evidence","claim","complete"]