[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7F3XHH0NMYPR2GWXKKK0SAM":3,"skill-same:01M7F3XHH0NMYPR2GWXKKK0SAM":47},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":25,"commitSha":26,"skillDirectory":27,"forksCount":28,"repositoryHtmlUrl":29,"repositoryHomepageUrl":6,"ownerHtmlUrl":30,"ownerBlogUrl":6,"ownerDescription":31,"updatedAt":32,"install":33,"resources":46},"01M7F3XHH0NMYPR2GWXKKK0SAM","regex-vs-llm-structured-text",null,"Decision framework for choosing between regex and LLM when parsing structured text — start with regex, add LLM only for low-confidence edge cases.","lza6\u002FArmor-piercing-system-prompts","lza6","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F128880206?v=4","01M7EB3K7WPX5T28EJ2NVHE6EG","Chinese-tingfeng","01M7EB3K97E1KQ16QWKJMFNGX8",11,"MIT","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F05-技能与代理开发\u002Fregex-vs-llm-structured-text\u002FSKILL.md",[],[],"2026-11-16 10:05:34",false,"---\nname: regex-vs-llm-structured-text\ndescription: Decision framework for choosing between regex and LLM when parsing structured text — start with regex, add LLM only for low-confidence edge cases.\norigin: ECC\n---\n\n# Regex vs LLM for Structured Text Parsing\n\nA practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.\n\n## When to Activate\n\n- Parsing structured text with repeating patterns (questions, forms, tables)\n- Deciding between regex and LLM for text extraction\n- Building hybrid pipelines that combine both approaches\n- Optimizing cost\u002Faccuracy tradeoffs in text processing\n\n## Decision Framework\n\n```\nIs the text format consistent and repeating?\n├── Yes (>90% follows a pattern) → Start with Regex\n│   ├── Regex handles 95%+ → Done, no LLM needed\n│   └── Regex handles \u003C95% → Add LLM for edge cases only\n└── No (free-form, highly variable) → Use LLM directly\n```\n\n## Architecture Pattern\n\n```\nSource Text\n    │\n    ▼\n[Regex Parser] ─── Extracts structure (95-98% accuracy)\n    │\n    ▼\n[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)\n    │\n    ▼\n[Confidence Scorer] ─── Flags low-confidence extractions\n    │\n    ├── High confidence (≥0.95) → Direct output\n    │\n    └── Low confidence (\u003C0.95) → [LLM Validator] → Output\n```\n\n## Implementation\n\n### 1. Regex Parser (Handles the Majority)\n\n```python\nimport re\nfrom dataclasses import dataclass\n\n@dataclass(frozen=True)\nclass ParsedItem:\n    id: str\n    text: str\n    choices: tuple[str, ...]\n    answer: str\n    confidence: float = 1.0\n\ndef parse_structured_text(content: str) -> list[ParsedItem]:\n    \"\"\"Parse structured text using regex patterns.\"\"\"\n    pattern = re.compile(\n        r\"(?P\u003Cid>\\d+)\\.\\s*(?P\u003Ctext>.+?)\\n\"\n        r\"(?P\u003Cchoices>(?:[A-D]\\..+?\\n)+)\"\n        r\"Answer:\\s*(?P\u003Canswer>[A-D])\",\n        re.MULTILINE | re.DOTALL,\n    )\n    items = []\n    for match in pattern.finditer(content):\n        choices = tuple(\n            c.strip() for c in re.findall(r\"[A-D]\\.\\s*(.+)\", match.group(\"choices\"))\n        )\n        items.append(ParsedItem(\n            id=match.group(\"id\"),\n            text=match.group(\"text\").strip(),\n            choices=choices,\n            answer=match.group(\"answer\"),\n        ))\n    return items\n```\n\n### 2. Confidence Scoring\n\nFlag items that may need LLM review:\n\n```python\n@dataclass(frozen=True)\nclass ConfidenceFlag:\n    item_id: str\n    score: float\n    reasons: tuple[str, ...]\n\ndef score_confidence(item: ParsedItem) -> ConfidenceFlag:\n    \"\"\"Score extraction confidence and flag issues.\"\"\"\n    reasons = []\n    score = 1.0\n\n    if len(item.choices) \u003C 3:\n        reasons.append(\"few_choices\")\n        score -= 0.3\n\n    if not item.answer:\n        reasons.append(\"missing_answer\")\n        score -= 0.5\n\n    if len(item.text) \u003C 10:\n        reasons.append(\"short_text\")\n        score -= 0.2\n\n    return ConfidenceFlag(\n        item_id=item.id,\n        score=max(0.0, score),\n        reasons=tuple(reasons),\n    )\n\ndef identify_low_confidence(\n    items: list[ParsedItem],\n    threshold: float = 0.95,\n) -> list[ConfidenceFlag]:\n    \"\"\"Return items below confidence threshold.\"\"\"\n    flags = [score_confidence(item) for item in items]\n    return [f for f in flags if f.score \u003C threshold]\n```\n\n### 3. LLM Validator (Edge Cases Only)\n\n```python\ndef validate_with_llm(\n    item: ParsedItem,\n    original_text: str,\n    client,\n) -> ParsedItem:\n    \"\"\"Use LLM to fix low-confidence extractions.\"\"\"\n    response = client.messages.create(\n        model=\"claude-haiku-4-5-20251001\",  # Cheapest model for validation\n        max_tokens=500,\n        messages=[{\n            \"role\": \"user\",\n            \"content\": (\n                f\"Extract the question, choices, and answer from this text.\\n\\n\"\n                f\"Text: {original_text}\\n\\n\"\n                f\"Current extraction: {item}\\n\\n\"\n                f\"Return corrected JSON if needed, or 'CORRECT' if accurate.\"\n            ),\n        }],\n    )\n    # Parse LLM response and return corrected item...\n    return corrected_item\n```\n\n### 4. Hybrid Pipeline\n\n```python\ndef process_document(\n    content: str,\n    *,\n    llm_client=None,\n    confidence_threshold: float = 0.95,\n) -> list[ParsedItem]:\n    \"\"\"Full pipeline: regex -> confidence check -> LLM for edge cases.\"\"\"\n    # Step 1: Regex extraction (handles 95-98%)\n    items = parse_structured_text(content)\n\n    # Step 2: Confidence scoring\n    low_confidence = identify_low_confidence(items, confidence_threshold)\n\n    if not low_confidence or llm_client is None:\n        return items\n\n    # Step 3: LLM validation (only for flagged items)\n    low_conf_ids = {f.item_id for f in low_confidence}\n    result = []\n    for item in items:\n        if item.id in low_conf_ids:\n            result.append(validate_with_llm(item, content, llm_client))\n        else:\n            result.append(item)\n\n    return result\n```\n\n## Real-World Metrics\n\nFrom a production quiz parsing pipeline (410 items):\n\n| Metric | Value |\n|--------|-------|\n| Regex success rate | 98.0% |\n| Low confidence items | 8 (2.0%) |\n| LLM calls needed | ~5 |\n| Cost savings vs all-LLM | ~95% |\n| Test coverage | 93% |\n\n## Best Practices\n\n- **Start with regex** — even imperfect regex gives you a baseline to improve\n- **Use confidence scoring** to programmatically identify what needs LLM help\n- **Use the cheapest LLM** for validation (Haiku-class models are sufficient)\n- **Never mutate** parsed items — return new instances from cleaning\u002Fvalidation steps\n- **TDD works well** for parsers — write tests for known patterns first, then edge cases\n- **Log metrics** (regex success rate, LLM call count) to track pipeline health\n\n## Anti-Patterns to Avoid\n\n- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)\n- Using regex for free-form, highly variable text (LLM is better here)\n- Skipping confidence scoring and hoping regex \"just works\"\n- Mutating parsed objects during cleaning\u002Fvalidation steps\n- Not testing edge cases (malformed input, missing fields, encoding issues)\n\n## When to Use\n\n- Quiz\u002Fexam question parsing\n- Form data extraction\n- Invoice\u002Freceipt processing\n- Document structure parsing (headers, sections, tables)\n- Any structured text with repeating patterns where cost matters\n","# Regex vs LLM for Structured Text Parsing\n\nA practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.\n\n## When to Activate\n\n- Parsing structured text with repeating patterns (questions, forms, tables)\n- Deciding between regex and LLM for text extraction\n- Building hybrid pipelines that combine both approaches\n- Optimizing cost\u002Faccuracy tradeoffs in text processing\n\n## Decision Framework\n\n```\nIs the text format consistent and repeating?\n├── Yes (>90% follows a pattern) → Start with Regex\n│   ├── Regex handles 95%+ → Done, no LLM needed\n│   └── Regex handles \u003C95% → Add LLM for edge cases only\n└── No (free-form, highly variable) → Use LLM directly\n```\n\n## Architecture Pattern\n\n```\nSource Text\n    │\n    ▼\n[Regex Parser] ─── Extracts structure (95-98% accuracy)\n    │\n    ▼\n[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)\n    │\n    ▼\n[Confidence Scorer] ─── Flags low-confidence extractions\n    │\n    ├── High confidence (≥0.95) → Direct output\n    │\n    └── Low confidence (\u003C0.95) → [LLM Validator] → Output\n```\n\n## Implementation\n\n### 1. Regex Parser (Handles the Majority)\n\n```python\nimport re\nfrom dataclasses import dataclass\n\n@dataclass(frozen=True)\nclass ParsedItem:\n    id: str\n    text: str\n    choices: tuple[str, ...]\n    answer: str\n    confidence: float = 1.0\n\ndef parse_structured_text(content: str) -> list[ParsedItem]:\n    \"\"\"Parse structured text using regex patterns.\"\"\"\n    pattern = re.compile(\n        r\"(?P\u003Cid>\\d+)\\.\\s*(?P\u003Ctext>.+?)\\n\"\n        r\"(?P\u003Cchoices>(?:[A-D]\\..+?\\n)+)\"\n        r\"Answer:\\s*(?P\u003Canswer>[A-D])\",\n        re.MULTILINE | re.DOTALL,\n    )\n    items = []\n    for match in pattern.finditer(content):\n        choices = tuple(\n            c.strip() for c in re.findall(r\"[A-D]\\.\\s*(.+)\", match.group(\"choices\"))\n        )\n        items.append(ParsedItem(\n            id=match.group(\"id\"),\n            text=match.group(\"text\").strip(),\n            choices=choices,\n            answer=match.group(\"answer\"),\n        ))\n    return items\n```\n\n### 2. Confidence Scoring\n\nFlag items that may need LLM review:\n\n```python\n@dataclass(frozen=True)\nclass ConfidenceFlag:\n    item_id: str\n    score: float\n    reasons: tuple[str, ...]\n\ndef score_confidence(item: ParsedItem) -> ConfidenceFlag:\n    \"\"\"Score extraction confidence and flag issues.\"\"\"\n    reasons = []\n    score = 1.0\n\n    if len(item.choices) \u003C 3:\n        reasons.append(\"few_choices\")\n        score -= 0.3\n\n    if not item.answer:\n        reasons.append(\"missing_answer\")\n        score -= 0.5\n\n    if len(item.text) \u003C 10:\n        reasons.append(\"short_text\")\n        score -= 0.2\n\n    return ConfidenceFlag(\n        item_id=item.id,\n        score=max(0.0, score),\n        reasons=tuple(reasons),\n    )\n\ndef identify_low_confidence(\n    items: list[ParsedItem],\n    threshold: float = 0.95,\n) -> list[ConfidenceFlag]:\n    \"\"\"Return items below confidence threshold.\"\"\"\n    flags = [score_confidence(item) for item in items]\n    return [f for f in flags if f.score \u003C threshold]\n```\n\n### 3. LLM Validator (Edge Cases Only)\n\n```python\ndef validate_with_llm(\n    item: ParsedItem,\n    original_text: str,\n    client,\n) -> ParsedItem:\n    \"\"\"Use LLM to fix low-confidence extractions.\"\"\"\n    response = client.messages.create(\n        model=\"claude-haiku-4-5-20251001\",  # Cheapest model for validation\n        max_tokens=500,\n        messages=[{\n            \"role\": \"user\",\n            \"content\": (\n                f\"Extract the question, choices, and answer from this text.\\n\\n\"\n                f\"Text: {original_text}\\n\\n\"\n                f\"Current extraction: {item}\\n\\n\"\n                f\"Return corrected JSON if needed, or 'CORRECT' if accurate.\"\n            ),\n        }],\n    )\n    # Parse LLM response and return corrected item...\n    return corrected_item\n```\n\n### 4. Hybrid Pipeline\n\n```python\ndef process_document(\n    content: str,\n    *,\n    llm_client=None,\n    confidence_threshold: float = 0.95,\n) -> list[ParsedItem]:\n    \"\"\"Full pipeline: regex -> confidence check -> LLM for edge cases.\"\"\"\n    # Step 1: Regex extraction (handles 95-98%)\n    items = parse_structured_text(content)\n\n    # Step 2: Confidence scoring\n    low_confidence = identify_low_confidence(items, confidence_threshold)\n\n    if not low_confidence or llm_client is None:\n        return items\n\n    # Step 3: LLM validation (only for flagged items)\n    low_conf_ids = {f.item_id for f in low_confidence}\n    result = []\n    for item in items:\n        if item.id in low_conf_ids:\n            result.append(validate_with_llm(item, content, llm_client))\n        else:\n            result.append(item)\n\n    return result\n```\n\n## Real-World Metrics\n\nFrom a production quiz parsing pipeline (410 items):\n\n| Metric | Value |\n|--------|-------|\n| Regex success rate | 98.0% |\n| Low confidence items | 8 (2.0%) |\n| LLM calls needed | ~5 |\n| Cost savings vs all-LLM | ~95% |\n| Test coverage | 93% |\n\n## Best Practices\n\n- **Start with regex** — even imperfect regex gives you a baseline to improve\n- **Use confidence scoring** to programmatically identify what needs LLM help\n- **Use the cheapest LLM** for validation (Haiku-class models are sufficient)\n- **Never mutate** parsed items — return new instances from cleaning\u002Fvalidation steps\n- **TDD works well** for parsers — write tests for known patterns first, then edge cases\n- **Log metrics** (regex success rate, LLM call count) to track pipeline health\n\n## Anti-Patterns to Avoid\n\n- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)\n- Using regex for free-form, highly variable text (LLM is better here)\n- Skipping confidence scoring and hoping regex \"just works\"\n- Mutating parsed objects during cleaning\u002Fvalidation steps\n- Not testing edge cases (malformed input, missing fields, encoding issues)\n\n## When to Use\n\n- Quiz\u002Fexam question parsing\n- Form data extraction\n- Invoice\u002Freceipt processing\n- Document structure parsing (headers, sections, tables)\n- Any structured text with repeating patterns where cost matters\n",{"name":5,"origin":24,"description":7},"ECC","main","d69096e239f2a89aabc2cd078c47664a9d3b1e11","05-技能与代理开发\u002Fregex-vs-llm-structured-text",3,"https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts","https:\u002F\u002Fgithub.com\u002Flza6","个人vx：Tf00798\r\n个人✈：@TFGY999","2026-10-08 14:29:22",{"source":8,"skillName":5,"commands":34,"prompt":44,"zipAvailable":45},[35,38,41],{"tool":36,"command":37},"npx","npx skills add lza6\u002FArmor-piercing-system-prompts --skill regex-vs-llm-structured-text",{"tool":39,"command":40},"bunx","bunx skills add lza6\u002FArmor-piercing-system-prompts --skill regex-vs-llm-structured-text",{"tool":42,"command":43},"pnpm","pnpm dlx skills add lza6\u002FArmor-piercing-system-prompts --skill regex-vs-llm-structured-text","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F05-技能与代理开发\u002Fregex-vs-llm-structured-text\u002FSKILL.md",true,[],[48,55,60,67,74,81],{"skillNo":49,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":52,"categories":53,"keywords":54,"publishedAt":19},"01M7FFG485KY83TD6HSC4YPCMC","obsidian-clipper-template-creator","Guide for creating templates for the Obsidian Web Clipper. Use when you want to create a new clipping template, understand available variables, or format clipped content.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobsidian-clipper-template-creator\u002FSKILL.md",[],[],{"skillNo":56,"name":50,"nameZh":6,"description":51,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":57,"categories":58,"keywords":59,"publishedAt":19},"01M7FFG0T6AZ0STEJPSB3CX84K","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobsidian-clipper-template-creator-antigravity-awesome-skills-main\u002FSKILL.md",[],[],{"skillNo":61,"name":62,"nameZh":6,"description":63,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":64,"categories":65,"keywords":66,"publishedAt":19},"01M7FFFY0QEM14SKWPDYGMXM2P","python-api","Programmatic Object-Detection-Metrics API guidance for BoundingBox, BoundingBoxes, Evaluator, enums, coordinate conversion, AP\u002FmAP semantics, and source-style import diagnostics.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002Fsub-skills\u002Fpython-api\u002FSKILL.md",[],[],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":19},"01M7FFFVZQZ2GJ0C0TGP7JHD20","file-evaluation","Folder-based PASCAL VOC AP\u002FmAP evaluation from ground-truth and detection text files.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002Fsub-skills\u002Ffile-evaluation\u002FSKILL.md",[],[],{"skillNo":75,"name":76,"nameZh":6,"description":77,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":80,"publishedAt":19},"01M7FFFSFNYSE77SJ6PE1CK2DA","object-detection-metrics","Guides legacy Object-Detection-Metrics PASCAL VOC AP\u002FmAP evaluation from detection text folders and direct Python API classes.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fobject-detection-metrics\u002FSKILL.md",[],[],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":87,"publishedAt":19},"01M7FFFP0GCK1RVXBCPQ8APBMT","nx-workspace-patterns","Configure and optimize Nx monorepo workspaces. Use when setting up Nx, configuring project boundaries, optimizing build caching, or implementing affected commands.","https:\u002F\u002Fgithub.com\u002Flza6\u002FArmor-piercing-system-prompts\u002Fblob\u002Fmain\u002F07-后端与架构\u002Fnx-workspace-patterns\u002FSKILL.md",[],[]]