[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M72Z3RC8SH5M2G9ENNHVPZT2":3,"skill-same:01M72Z3RC8SH5M2G9ENNHVPZT2":101},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":6,"ownerHtmlUrl":29,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":30,"install":31,"resources":44},"01M72Z3RC8SH5M2G9ENNHVPZT2","book-sft-pipeline",null,"This skill should be used when the user asks to \"fine-tune on books\", \"create SFT dataset\", \"train style model\", \"extract ePub text\", or mentions style transfer, LoRA training, book segmentation, or author voice replication.","enigmaicon-eng\u002FAI-Enterprise-OS","enigmaicon-eng","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F269773191?v=4","01M72YXXYN7Q7BYRHV8RAVYNWD","01M72YXXZPW35H56D6F1XSK84D",2,"MIT","https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: book-sft-pipeline\ndescription: This skill should be used when the user asks to \"fine-tune on books\", \"create SFT dataset\", \"train style model\", \"extract ePub text\", or mentions style transfer, LoRA training, book segmentation, or author voice replication.\nversion: 2.0.0\n---\n\n# Book SFT Pipeline\n\nA complete system for converting books into SFT datasets and training style-transfer models. This skill teaches the pipeline from raw ePub to a model that writes in any author's voice.\n\n## When to Activate\n\nActivate this skill when:\n- Building fine-tuning datasets from literary works\n- Creating author-voice or style-transfer models\n- Preparing training data for Tinker or similar SFT platforms\n- Designing text segmentation pipelines for long-form content\n- Training small models (8B or less) on limited data\n\n## Core Concepts\n\n### The Three Pillars of Book SFT\n\n**1. Intelligent Segmentation**\nText chunks must be semantically coherent. Breaking mid-sentence teaches the model to produce fragmented output. Target: 150-400 words per chunk, always at natural boundaries.\n\n**2. Diverse Instruction Generation**\nUse multiple prompt templates and system prompts to prevent overfitting. A single prompt style leads to memorization. Use 15+ prompt templates with 5+ system prompts.\n\n**3. Style Over Content**\nThe goal is learning the author's rhythm and vocabulary patterns, not memorizing plots. Synthetic instructions describe what happens without quoting the text.\n\n## Pipeline Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│                    ORCHESTRATOR AGENT                           │\n│  Coordinates pipeline phases, manages state, handles failures   │\n└──────────────────────┬──────────────────────────────────────────┘\n                       │\n       ┌───────────────┼───────────────┬───────────────┐\n       ▼               ▼               ▼               ▼\n┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐\n│  EXTRACTION  │ │ SEGMENTATION │ │  INSTRUCTION │ │   DATASET    │\n│    AGENT     │ │    AGENT     │ │    AGENT     │ │   BUILDER    │\n│ ePub → Text  │ │ Text → Chunks│ │ Chunks →     │ │ Pairs →      │\n│              │ │ 150-400 words│ │ Prompts      │ │ JSONL        │\n└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘\n                       │\n       ┌───────────────┴───────────────┐\n       ▼                               ▼\n┌──────────────┐               ┌──────────────┐\n│   TRAINING   │               │  VALIDATION  │\n│    AGENT     │               │    AGENT     │\n│ LoRA on      │               │ AI detector  │\n│ Tinker       │               │ Originality  │\n└──────────────┘               └──────────────┘\n```\n\n## Phase 1: Text Extraction\n\n### Critical Rules\n1. **Always source ePub over PDF** - OCR errors become learned patterns\n2. **Use paragraph-level extraction** - Extract from `\u003Cp>` tags to preserve breaks\n3. **Remove front\u002Fback matter** - Copyright and TOC pollute the dataset\n\n```python\n# Extract text from ePub paragraphs\nfrom epub2 import EPub\nfrom bs4 import BeautifulSoup\n\ndef extract_epub(path):\n    book = EPub(path)\n    chapters = []\n    for item in book.flow:\n        html = book.get_chapter(item.id)\n        soup = BeautifulSoup(html, 'html.parser')\n        paragraphs = [p.get_text().strip() for p in soup.find_all('p')]\n        chapters.append('\\n\\n'.join(p for p in paragraphs if p))\n    return '\\n\\n'.join(chapters)\n```\n\n## Phase 2: Intelligent Segmentation\n\n### Smaller Chunks + Overlap\n\nSmaller chunks (150-400 words) produce more training examples and better style transfer than larger chunks (250-650).\n\n```python\ndef segment(text, min_words=150, max_words=400):\n    paragraphs = text.split('\\n\\n')\n    chunks, buffer, buffer_words = [], [], 0\n    \n    for para in paragraphs:\n        words = len(para.split())\n        if buffer_words + words > max_words and buffer_words >= min_words:\n            chunks.append('\\n\\n'.join(buffer))\n            # Keep last paragraph for overlap\n            buffer = [buffer[-1], para] if buffer else [para]\n            buffer_words = sum(len(p.split()) for p in buffer)\n        else:\n            buffer.append(para)\n            buffer_words += words\n    \n    if buffer:\n        chunks.append('\\n\\n'.join(buffer))\n    return chunks\n```\n\n### Expected Results\n\nFor an 86,000-word book:\n- Old method (250-650 words): ~150 chunks\n- New method (150-400 + overlap): ~300 chunks\n- With 2 variants per chunk: 600+ training examples\n\n## Phase 3: Diverse Instruction Generation\n\n### The Key Insight\n\nUsing a single prompt template causes memorization. Diverse templates teach the underlying style.\n\n```python\nSYSTEM_PROMPTS = [\n    \"You are an expert creative writer capable of emulating specific literary styles.\",\n    \"You are a literary writer with deep knowledge of classic prose styles.\",\n    \"You are a creative writer skilled at emulating distinctive authorial voices.\",\n    \"You write prose that captures the essence of modernist literature.\",\n    \"You are a talented writer who can channel classic American authors.\",\n]\n\nPROMPT_TEMPLATES = [\n    \"Write a passage in the style of {author}: {desc}\",\n    \"Channel {author}'s voice to write about: {desc}\",\n    \"In {author}'s distinctive prose style, describe: {desc}\",\n    \"Write this scene as {author} would have: {desc}\",\n    \"Using {author}'s repetitive technique, describe: {desc}\",\n    \"Capture the rhythm of {author} in this passage: {desc}\",\n    \"Write like {author}: {desc}\",\n    \"In the voice of {author}, write: {desc}\",\n    \"This is a literary exercise. Write like {author}: {desc}\",\n    \"Can you write in {author}'s style? {desc}\",\n]\n```\n\n### Instruction Generation\n\n```python\nINSTRUCTION_PROMPT = \"\"\"Describe what is happening in this excerpt in 2-3 sentences.\nFocus on: characters present, actions, emotions, setting.\nDo NOT quote the text directly.\n\nExcerpt:\n{text}\n\"\"\"\n\n# Use a fast, cheap LLM (e.g., Gemini Flash)\ninstruction = llm_call(INSTRUCTION_PROMPT.format(text=chunk))\n```\n\n## Phase 4: Dataset Construction\n\n### Message Format\n\n```json\n{\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are an expert creative writer...\"},\n        {\"role\": \"user\", \"content\": \"Write in the style of Author: Scene description...\"},\n        {\"role\": \"assistant\", \"content\": \"The actual book text from chunk...\"}\n    ]\n}\n```\n\n### Multiple Variants Per Chunk\n\n```python\ndef build_examples(chunk, instruction, author, variants=2):\n    examples = []\n    for i in range(variants):\n        system = SYSTEM_PROMPTS[i % len(SYSTEM_PROMPTS)]\n        template = PROMPT_TEMPLATES[(chunk.id + i) % len(PROMPT_TEMPLATES)]\n        user = template.format(author=author, desc=instruction)\n        examples.append({\"messages\": [\n            {\"role\": \"system\", \"content\": system},\n            {\"role\": \"user\", \"content\": user},\n            {\"role\": \"assistant\", \"content\": chunk.text}\n        ]})\n    return examples\n```\n\n## Phase 5: LoRA Training on Tinker\n\n### Configuration\n\n```python\nCONFIG = {\n    \"model_name\": \"Qwen\u002FQwen3-8B-Base\",  # Base, not instruct\n    \"lora_rank\": 32,                      # 352MB adapter\n    \"learning_rate\": 5e-4,                # Higher for LoRA\n    \"batch_size\": 4,\n    \"epochs\": 3,\n}\n```\n\n### Why Base Model?\n\nUse **base** (pretrained) models, not instruction-tuned versions:\n- Base models are more malleable for new styles\n- Instruct models have patterns that resist overwriting\n- Style is a low-level pattern that base models capture better\n\n### Training Loop\n\n```python\nimport tinker\nfrom tinker import types\n\ntraining_client = await service_client.create_lora_training_client_async(\n    base_model=\"Qwen\u002FQwen3-8B-Base\",\n    rank=32\n)\n\nfor epoch in range(3):\n    for batch in batches:\n        await training_client.forward_backward_async(batch, loss_fn=\"cross_entropy\")\n        await training_client.optim_step_async(types.AdamParams(learning_rate=5e-4))\n\nresult = await training_client.save_weights_for_sampler_async(name=\"final\")\n```\n\n## Phase 6: Validation\n\n### Modern Scenario Test\n\nTest with scenarios that couldn't exist in the original book:\n\n```python\nTEST_PROMPTS = [\n    \"Write about a barista making lattes\",\n    \"Describe lovers communicating through text messages\",\n    \"Write about someone anxious about climate change\",\n]\n```\n\nIf the model applies style markers to modern scenarios, it learned **style**, not **content**.\n\n### Originality Verification\n\n```bash\n# Search training data for output phrases\ngrep \"specific phrase from output\" dataset.jsonl\n# Should return: No matches\n```\n\n### AI Detector Testing\n\nTest outputs with GPTZero, Pangram, or ZeroGPT.\n\n## Known Issues and Solutions\n\n### Character Name Leakage\n\n**Symptom**: Model uses original character names in new scenarios.\n**Cause**: Limited name diversity from one book.\n**Solution**: Train on multiple books or add synthetic examples.\n\n### Model Parrots Exact Phrases\n\n**Symptom**: Outputs contain exact sentences from training data.\n**Cause**: Too few prompt variations or too many epochs.\n**Solution**: Use 15+ templates, limit to 3 epochs.\n\n### Fragmented Outputs\n\n**Symptom**: Sentences feel incomplete.\n**Cause**: Poor segmentation breaking mid-thought.\n**Solution**: Always break at paragraph boundaries.\n\n## Guidelines\n\n1. **Always source ePub over PDF** - OCR errors become learned patterns\n2. **Never break mid-sentence** - Boundaries must be grammatically complete\n3. **Use diverse prompts** - 15+ templates, 5+ system prompts\n4. **Use base models** - Not instruct versions\n5. **Use smaller chunks** - 150-400 words for more examples\n6. **Reserve test set** - 50 examples minimum\n7. **Test on modern scenarios** - Proves style transfer vs memorization\n8. **Verify originality** - Grep training data for output phrases\n\n## Expected Results\n\n| Metric | Value |\n|--------|-------|\n| Training examples | 500-1000 per book |\n| Model | Qwen\u002FQwen3-8B-Base |\n| LoRA rank | 32 |\n| Adapter size | ~350 MB |\n| Training time | ~15 min |\n| Loss reduction | 90%+ |\n| Style transfer success | ~50% perfect |\n\n## Cost Estimate\n\n| Component | Cost |\n|-----------|------|\n| LLM (instruction generation) | ~$0.50 |\n| Tinker training (15 min) | ~$1.50 |\n| **Total** | **~$2.00** |\n\n## Integration with Context Engineering Skills\n\nThis example applies several skills from the Agent Skills for Context Engineering collection:\n\n### project-development\nThe pipeline follows the staged, idempotent architecture pattern:\n- **Acquire**: Extract text from ePub\n- **Prepare**: Segment into training chunks\n- **Process**: Generate synthetic instructions\n- **Parse**: Build message format\n- **Render**: Output Tinker-compatible JSONL\n- **Train**: LoRA fine-tuning\n- **Validate**: Modern scenario testing\n\nEach phase is resumable and produces intermediate artifacts for debugging.\n\n### context-compression\nSegmentation is a form of context compression for training. The core insight from context-compression applies: information density matters more than information quantity. Smaller, coherent chunks (150-400 words) produce better style transfer than larger, diluted chunks.\n\nThe two-tier strategy mirrors context compression evaluation:\n- Tier 1: Fast, deterministic compression\n- Tier 2: LLM-assisted for edge cases\n\n### multi-agent-patterns\nThe pipeline uses the **supervisor\u002Forchestrator** pattern:\n- Orchestrator coordinates phases and manages state\n- Specialized agents (Extraction, Segmentation, Instruction, Builder) have isolated contexts\n- Each agent receives only the information needed for its task\n\nThis matches the principle that sub-agents exist primarily to isolate context rather than simulate roles.\n\n### evaluation\nValidation follows the **end-state evaluation** pattern:\n- Functional testing: Does output match expected style markers?\n- Originality verification: Is content genuinely generated?\n- External validation: AI detector scores\n\nThe \"modern scenario\" test is a form of out-of-distribution evaluation that proves generalization.\n\n### context-fundamentals\nPrompt diversity prevents attention collapse on single patterns. When training with identical prompt structures, the model memorizes the instruction-response mapping. Diverse templates force attention across the style patterns themselves.\n\n## References\n\nInternal references:\n- [Segmentation Strategies](.\u002Freferences\u002Fsegmentation-strategies.md) - Text chunking patterns\n- [Tinker Format Specification](.\u002Freferences\u002Ftinker-format.md) - Datum structure\n- [Tinker API Documentation](.\u002Freferences\u002Ftinker.txt) - Full API reference\n\nRelated skills from Agent Skills for Context Engineering:\n- project-development - Pipeline architecture patterns\n- context-compression - Compression strategies  \n- multi-agent-patterns - Agent coordination\n- evaluation - Evaluation frameworks\n- context-fundamentals - Attention and information density\n\nExternal resources:\n- [Research Paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2510.13939) - Chakrabarty et al. 2025\n- [Dataset on Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FMuratcanKoylan\u002Fgertrude-stein-style-sft)\n- [Gertrude Stein Case Study](.\u002Fexamples\u002Fgertrude-stein\u002F) - Complete working example\n\n---\n\n## Skill Metadata\n\n**Created**: 2025-12-26\n**Last Updated**: 2025-12-28\n**Author**: Muratcan Koylan\n**Version**: 2.0.0\n**Standalone**: Yes (separate from main context-engineering collection)\n","# Book SFT Pipeline\n\nA complete system for converting books into SFT datasets and training style-transfer models. This skill teaches the pipeline from raw ePub to a model that writes in any author's voice.\n\n## When to Activate\n\nActivate this skill when:\n- Building fine-tuning datasets from literary works\n- Creating author-voice or style-transfer models\n- Preparing training data for Tinker or similar SFT platforms\n- Designing text segmentation pipelines for long-form content\n- Training small models (8B or less) on limited data\n\n## Core Concepts\n\n### The Three Pillars of Book SFT\n\n**1. Intelligent Segmentation**\nText chunks must be semantically coherent. Breaking mid-sentence teaches the model to produce fragmented output. Target: 150-400 words per chunk, always at natural boundaries.\n\n**2. Diverse Instruction Generation**\nUse multiple prompt templates and system prompts to prevent overfitting. A single prompt style leads to memorization. Use 15+ prompt templates with 5+ system prompts.\n\n**3. Style Over Content**\nThe goal is learning the author's rhythm and vocabulary patterns, not memorizing plots. Synthetic instructions describe what happens without quoting the text.\n\n## Pipeline Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│                    ORCHESTRATOR AGENT                           │\n│  Coordinates pipeline phases, manages state, handles failures   │\n└──────────────────────┬──────────────────────────────────────────┘\n                       │\n       ┌───────────────┼───────────────┬───────────────┐\n       ▼               ▼               ▼               ▼\n┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐\n│  EXTRACTION  │ │ SEGMENTATION │ │  INSTRUCTION │ │   DATASET    │\n│    AGENT     │ │    AGENT     │ │    AGENT     │ │   BUILDER    │\n│ ePub → Text  │ │ Text → Chunks│ │ Chunks →     │ │ Pairs →      │\n│              │ │ 150-400 words│ │ Prompts      │ │ JSONL        │\n└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘\n                       │\n       ┌───────────────┴───────────────┐\n       ▼                               ▼\n┌──────────────┐               ┌──────────────┐\n│   TRAINING   │               │  VALIDATION  │\n│    AGENT     │               │    AGENT     │\n│ LoRA on      │               │ AI detector  │\n│ Tinker       │               │ Originality  │\n└──────────────┘               └──────────────┘\n```\n\n## Phase 1: Text Extraction\n\n### Critical Rules\n1. **Always source ePub over PDF** - OCR errors become learned patterns\n2. **Use paragraph-level extraction** - Extract from `\u003Cp>` tags to preserve breaks\n3. **Remove front\u002Fback matter** - Copyright and TOC pollute the dataset\n\n```python\n# Extract text from ePub paragraphs\nfrom epub2 import EPub\nfrom bs4 import BeautifulSoup\n\ndef extract_epub(path):\n    book = EPub(path)\n    chapters = []\n    for item in book.flow:\n        html = book.get_chapter(item.id)\n        soup = BeautifulSoup(html, 'html.parser')\n        paragraphs = [p.get_text().strip() for p in soup.find_all('p')]\n        chapters.append('\\n\\n'.join(p for p in paragraphs if p))\n    return '\\n\\n'.join(chapters)\n```\n\n## Phase 2: Intelligent Segmentation\n\n### Smaller Chunks + Overlap\n\nSmaller chunks (150-400 words) produce more training examples and better style transfer than larger chunks (250-650).\n\n```python\ndef segment(text, min_words=150, max_words=400):\n    paragraphs = text.split('\\n\\n')\n    chunks, buffer, buffer_words = [], [], 0\n    \n    for para in paragraphs:\n        words = len(para.split())\n        if buffer_words + words > max_words and buffer_words >= min_words:\n            chunks.append('\\n\\n'.join(buffer))\n            # Keep last paragraph for overlap\n            buffer = [buffer[-1], para] if buffer else [para]\n            buffer_words = sum(len(p.split()) for p in buffer)\n        else:\n            buffer.append(para)\n            buffer_words += words\n    \n    if buffer:\n        chunks.append('\\n\\n'.join(buffer))\n    return chunks\n```\n\n### Expected Results\n\nFor an 86,000-word book:\n- Old method (250-650 words): ~150 chunks\n- New method (150-400 + overlap): ~300 chunks\n- With 2 variants per chunk: 600+ training examples\n\n## Phase 3: Diverse Instruction Generation\n\n### The Key Insight\n\nUsing a single prompt template causes memorization. Diverse templates teach the underlying style.\n\n```python\nSYSTEM_PROMPTS = [\n    \"You are an expert creative writer capable of emulating specific literary styles.\",\n    \"You are a literary writer with deep knowledge of classic prose styles.\",\n    \"You are a creative writer skilled at emulating distinctive authorial voices.\",\n    \"You write prose that captures the essence of modernist literature.\",\n    \"You are a talented writer who can channel classic American authors.\",\n]\n\nPROMPT_TEMPLATES = [\n    \"Write a passage in the style of {author}: {desc}\",\n    \"Channel {author}'s voice to write about: {desc}\",\n    \"In {author}'s distinctive prose style, describe: {desc}\",\n    \"Write this scene as {author} would have: {desc}\",\n    \"Using {author}'s repetitive technique, describe: {desc}\",\n    \"Capture the rhythm of {author} in this passage: {desc}\",\n    \"Write like {author}: {desc}\",\n    \"In the voice of {author}, write: {desc}\",\n    \"This is a literary exercise. Write like {author}: {desc}\",\n    \"Can you write in {author}'s style? {desc}\",\n]\n```\n\n### Instruction Generation\n\n```python\nINSTRUCTION_PROMPT = \"\"\"Describe what is happening in this excerpt in 2-3 sentences.\nFocus on: characters present, actions, emotions, setting.\nDo NOT quote the text directly.\n\nExcerpt:\n{text}\n\"\"\"\n\n# Use a fast, cheap LLM (e.g., Gemini Flash)\ninstruction = llm_call(INSTRUCTION_PROMPT.format(text=chunk))\n```\n\n## Phase 4: Dataset Construction\n\n### Message Format\n\n```json\n{\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are an expert creative writer...\"},\n        {\"role\": \"user\", \"content\": \"Write in the style of Author: Scene description...\"},\n        {\"role\": \"assistant\", \"content\": \"The actual book text from chunk...\"}\n    ]\n}\n```\n\n### Multiple Variants Per Chunk\n\n```python\ndef build_examples(chunk, instruction, author, variants=2):\n    examples = []\n    for i in range(variants):\n        system = SYSTEM_PROMPTS[i % len(SYSTEM_PROMPTS)]\n        template = PROMPT_TEMPLATES[(chunk.id + i) % len(PROMPT_TEMPLATES)]\n        user = template.format(author=author, desc=instruction)\n        examples.append({\"messages\": [\n            {\"role\": \"system\", \"content\": system},\n            {\"role\": \"user\", \"content\": user},\n            {\"role\": \"assistant\", \"content\": chunk.text}\n        ]})\n    return examples\n```\n\n## Phase 5: LoRA Training on Tinker\n\n### Configuration\n\n```python\nCONFIG = {\n    \"model_name\": \"Qwen\u002FQwen3-8B-Base\",  # Base, not instruct\n    \"lora_rank\": 32,                      # 352MB adapter\n    \"learning_rate\": 5e-4,                # Higher for LoRA\n    \"batch_size\": 4,\n    \"epochs\": 3,\n}\n```\n\n### Why Base Model?\n\nUse **base** (pretrained) models, not instruction-tuned versions:\n- Base models are more malleable for new styles\n- Instruct models have patterns that resist overwriting\n- Style is a low-level pattern that base models capture better\n\n### Training Loop\n\n```python\nimport tinker\nfrom tinker import types\n\ntraining_client = await service_client.create_lora_training_client_async(\n    base_model=\"Qwen\u002FQwen3-8B-Base\",\n    rank=32\n)\n\nfor epoch in range(3):\n    for batch in batches:\n        await training_client.forward_backward_async(batch, loss_fn=\"cross_entropy\")\n        await training_client.optim_step_async(types.AdamParams(learning_rate=5e-4))\n\nresult = await training_client.save_weights_for_sampler_async(name=\"final\")\n```\n\n## Phase 6: Validation\n\n### Modern Scenario Test\n\nTest with scenarios that couldn't exist in the original book:\n\n```python\nTEST_PROMPTS = [\n    \"Write about a barista making lattes\",\n    \"Describe lovers communicating through text messages\",\n    \"Write about someone anxious about climate change\",\n]\n```\n\nIf the model applies style markers to modern scenarios, it learned **style**, not **content**.\n\n### Originality Verification\n\n```bash\n# Search training data for output phrases\ngrep \"specific phrase from output\" dataset.jsonl\n# Should return: No matches\n```\n\n### AI Detector Testing\n\nTest outputs with GPTZero, Pangram, or ZeroGPT.\n\n## Known Issues and Solutions\n\n### Character Name Leakage\n\n**Symptom**: Model uses original character names in new scenarios.\n**Cause**: Limited name diversity from one book.\n**Solution**: Train on multiple books or add synthetic examples.\n\n### Model Parrots Exact Phrases\n\n**Symptom**: Outputs contain exact sentences from training data.\n**Cause**: Too few prompt variations or too many epochs.\n**Solution**: Use 15+ templates, limit to 3 epochs.\n\n### Fragmented Outputs\n\n**Symptom**: Sentences feel incomplete.\n**Cause**: Poor segmentation breaking mid-thought.\n**Solution**: Always break at paragraph boundaries.\n\n## Guidelines\n\n1. **Always source ePub over PDF** - OCR errors become learned patterns\n2. **Never break mid-sentence** - Boundaries must be grammatically complete\n3. **Use diverse prompts** - 15+ templates, 5+ system prompts\n4. **Use base models** - Not instruct versions\n5. **Use smaller chunks** - 150-400 words for more examples\n6. **Reserve test set** - 50 examples minimum\n7. **Test on modern scenarios** - Proves style transfer vs memorization\n8. **Verify originality** - Grep training data for output phrases\n\n## Expected Results\n\n| Metric | Value |\n|--------|-------|\n| Training examples | 500-1000 per book |\n| Model | Qwen\u002FQwen3-8B-Base |\n| LoRA rank | 32 |\n| Adapter size | ~350 MB |\n| Training time | ~15 min |\n| Loss reduction | 90%+ |\n| Style transfer success | ~50% perfect |\n\n## Cost Estimate\n\n| Component | Cost |\n|-----------|------|\n| LLM (instruction generation) | ~$0.50 |\n| Tinker training (15 min) | ~$1.50 |\n| **Total** | **~$2.00** |\n\n## Integration with Context Engineering Skills\n\nThis example applies several skills from the Agent Skills for Context Engineering collection:\n\n### project-development\nThe pipeline follows the staged, idempotent architecture pattern:\n- **Acquire**: Extract text from ePub\n- **Prepare**: Segment into training chunks\n- **Process**: Generate synthetic instructions\n- **Parse**: Build message format\n- **Render**: Output Tinker-compatible JSONL\n- **Train**: LoRA fine-tuning\n- **Validate**: Modern scenario testing\n\nEach phase is resumable and produces intermediate artifacts for debugging.\n\n### context-compression\nSegmentation is a form of context compression for training. The core insight from context-compression applies: information density matters more than information quantity. Smaller, coherent chunks (150-400 words) produce better style transfer than larger, diluted chunks.\n\nThe two-tier strategy mirrors context compression evaluation:\n- Tier 1: Fast, deterministic compression\n- Tier 2: LLM-assisted for edge cases\n\n### multi-agent-patterns\nThe pipeline uses the **supervisor\u002Forchestrator** pattern:\n- Orchestrator coordinates phases and manages state\n- Specialized agents (Extraction, Segmentation, Instruction, Builder) have isolated contexts\n- Each agent receives only the information needed for its task\n\nThis matches the principle that sub-agents exist primarily to isolate context rather than simulate roles.\n\n### evaluation\nValidation follows the **end-state evaluation** pattern:\n- Functional testing: Does output match expected style markers?\n- Originality verification: Is content genuinely generated?\n- External validation: AI detector scores\n\nThe \"modern scenario\" test is a form of out-of-distribution evaluation that proves generalization.\n\n### context-fundamentals\nPrompt diversity prevents attention collapse on single patterns. When training with identical prompt structures, the model memorizes the instruction-response mapping. Diverse templates force attention across the style patterns themselves.\n\n## References\n\nInternal references:\n- [Segmentation Strategies](.\u002Freferences\u002Fsegmentation-strategies.md) - Text chunking patterns\n- [Tinker Format Specification](.\u002Freferences\u002Ftinker-format.md) - Datum structure\n- [Tinker API Documentation](.\u002Freferences\u002Ftinker.txt) - Full API reference\n\nRelated skills from Agent Skills for Context Engineering:\n- project-development - Pipeline architecture patterns\n- context-compression - Compression strategies  \n- multi-agent-patterns - Agent coordination\n- evaluation - Evaluation frameworks\n- context-fundamentals - Attention and information density\n\nExternal resources:\n- [Research Paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2510.13939) - Chakrabarty et al. 2025\n- [Dataset on Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FMuratcanKoylan\u002Fgertrude-stein-style-sft)\n- [Gertrude Stein Case Study](.\u002Fexamples\u002Fgertrude-stein\u002F) - Complete working example\n\n---\n\n## Skill Metadata\n\n**Created**: 2025-12-26\n**Last Updated**: 2025-12-28\n**Author**: Muratcan Koylan\n**Version**: 2.0.0\n**Standalone**: Yes (separate from main context-engineering collection)\n",{"name":5,"version":23,"description":7},"2.0.0","main","a4b48842f3e48549adf3dc335a8837c66da99497","examples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline",0,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS","https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng","2026-05-17 16:00:01",{"source":8,"skillName":5,"commands":32,"prompt":42,"zipAvailable":43},[33,36,39],{"tool":34,"command":35},"npx","npx skills add enigmaicon-eng\u002FAI-Enterprise-OS --skill book-sft-pipeline",{"tool":37,"command":38},"bunx","bunx skills add enigmaicon-eng\u002FAI-Enterprise-OS --skill book-sft-pipeline",{"tool":40,"command":41},"pnpm","pnpm dlx skills add enigmaicon-eng\u002FAI-Enterprise-OS --skill book-sft-pipeline","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002FSKILL.md",true,[45,51,57,61,65,71,75,79,83,87,91,95],{"path":46,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":49,"sourceHtmlUrl":50},"examples\u002Fgertrude-stein\u002Fdataset_sample.jsonl","OTHER","其他",3968,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Fexamples\u002Fgertrude-stein\u002Fdataset_sample.jsonl",{"path":52,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":55,"sourceHtmlUrl":56},"examples\u002Fgertrude-stein\u002Fpangram\u002FScreenshot 2025-12-27 at 3.05.04 AM.png","ASSET","资产文件",206831,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Fexamples\u002Fgertrude-stein\u002Fpangram\u002FScreenshot 2025-12-27 at 3.05.04 AM.png",{"path":58,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":59,"sourceHtmlUrl":60},"examples\u002Fgertrude-stein\u002Fpangram\u002FScreenshot 2025-12-27 at 3.05.36 AM.png",137345,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Fexamples\u002Fgertrude-stein\u002Fpangram\u002FScreenshot 2025-12-27 at 3.05.36 AM.png",{"path":62,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":63,"sourceHtmlUrl":64},"examples\u002Fgertrude-stein\u002Fpangram\u002FScreenshot 2025-12-27 at 3.07.18 AM.png",132416,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Fexamples\u002Fgertrude-stein\u002Fpangram\u002FScreenshot 2025-12-27 at 3.07.18 AM.png",{"path":66,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":69,"sourceHtmlUrl":70},"examples\u002Fgertrude-stein\u002FREADME.md","REFERENCE","参考文档",5460,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Fexamples\u002Fgertrude-stein\u002FREADME.md",{"path":72,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":73,"sourceHtmlUrl":74},"examples\u002Fgertrude-stein\u002Fsample_outputs.md",3564,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Fexamples\u002Fgertrude-stein\u002Fsample_outputs.md",{"path":76,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":77,"sourceHtmlUrl":78},"examples\u002Fgertrude-stein\u002Ftraining_config.json",2814,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Fexamples\u002Fgertrude-stein\u002Ftraining_config.json",{"path":80,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":81,"sourceHtmlUrl":82},"README.md",2421,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002FREADME.md",{"path":84,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":85,"sourceHtmlUrl":86},"references\u002Fsegmentation-strategies.md",10817,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Freferences\u002Fsegmentation-strategies.md",{"path":88,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":89,"sourceHtmlUrl":90},"references\u002Ftinker-format.md",6068,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Freferences\u002Ftinker-format.md",{"path":92,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":93,"sourceHtmlUrl":94},"references\u002Ftinker.txt",146022,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Freferences\u002Ftinker.txt",{"path":96,"resourceType":97,"resourceTypeDesc":98,"sizeBytes":99,"sourceHtmlUrl":100},"scripts\u002Fpipeline_example.py","SCRIPT","脚本",6695,"https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fagent-skills\u002Fexamples\u002Fbook-sft-pipeline\u002Fscripts\u002Fpipeline_example.py",[102,109,116,123,130,137],{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":106,"categories":107,"keywords":108,"publishedAt":18},"01M72ZGWW03JSE1T3TKK5NZ11Y","writing-skills","Use when creating new skills, editing existing skills, or verifying skills work before deployment","https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fsuperpowers\u002Fskills\u002Fwriting-skills\u002FSKILL.md",[],[],{"skillNo":110,"name":111,"nameZh":6,"description":112,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":113,"categories":114,"keywords":115,"publishedAt":18},"01M72ZGT3FJT7QRM1VNQRKR0XD","writing-plans","Use when you have a spec or requirements for a multi-step task, before touching code","https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fsuperpowers\u002Fskills\u002Fwriting-plans\u002FSKILL.md",[],[],{"skillNo":117,"name":118,"nameZh":6,"description":119,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":120,"categories":121,"keywords":122,"publishedAt":18},"01M72ZGQ69DCF0SBP9JXFPWKJS","verification-before-completion","Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always","https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fsuperpowers\u002Fskills\u002Fverification-before-completion\u002FSKILL.md",[],[],{"skillNo":124,"name":125,"nameZh":6,"description":126,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":127,"categories":128,"keywords":129,"publishedAt":18},"01M72ZGNF126QT0MQ2D3DNGZRN","using-superpowers","Use when starting any conversation - establishes how to find and use skills, requiring Skill tool invocation before ANY response including clarifying questions","https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fsuperpowers\u002Fskills\u002Fusing-superpowers\u002FSKILL.md",[],[],{"skillNo":131,"name":132,"nameZh":6,"description":133,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":134,"categories":135,"keywords":136,"publishedAt":18},"01M72ZGHVJ9S0YXMNBHQGQ95BY","using-git-worktrees","Use when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback","https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fsuperpowers\u002Fskills\u002Fusing-git-worktrees\u002FSKILL.md",[],[],{"skillNo":138,"name":139,"nameZh":6,"description":140,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":141,"categories":142,"keywords":143,"publishedAt":18},"01M72ZGFNZCHFY2FS3ZD9MCYV9","test-driven-development","Use when implementing any feature or bugfix, before writing implementation code","https:\u002F\u002Fgithub.com\u002Fenigmaicon-eng\u002FAI-Enterprise-OS\u002Fblob\u002Fmain\u002Fexamples\u002Fsuperpowers\u002Fskills\u002Ftest-driven-development\u002FSKILL.md",[],[]]