[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M72C53MWT83GY85SCMYRJ2GX":3,"skill-same:01M72C53MWT83GY85SCMYRJ2GX":82},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":21,"frontMatter":22,"refName":42,"commitSha":43,"skillDirectory":44,"forksCount":13,"repositoryHtmlUrl":45,"repositoryHomepageUrl":6,"ownerHtmlUrl":46,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":47,"install":48,"resources":61},"01M72C53MWT83GY85SCMYRJ2GX","grpo-rl-training",null,"Expert guidance for GRPO\u002FRL fine-tuning with TRL for reasoning and task-specific model training","Sansar35\u002Fclaude-agent","Sansar35","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F145020281?v=4","01M724CQ2SCC86MGNSD87D0D4K","01M724CQ3965G9WJ3QXE3AJ5YY",1,"MIT","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fgrpo-rl-training\u002FSKILL.md",[],[],"2026-11-10 18:39:20",false,"---\nname: grpo-rl-training\ndescription: Expert guidance for GRPO\u002FRL fine-tuning with TRL for reasoning and task-specific model training\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Post-Training, Reinforcement Learning, GRPO, TRL, RLHF, Reward Modeling, Reasoning, DPO, PPO, Structured Output]\ndependencies: [transformers>=4.47.0, trl>=0.14.0, datasets>=3.2.0, peft>=0.14.0, torch]\n---\n\n# GRPO\u002FRL Training with TRL\n\nExpert-level guidance for implementing Group Relative Policy Optimization (GRPO) using the Transformer Reinforcement Learning (TRL) library. This skill provides battle-tested patterns, critical insights, and production-ready workflows for fine-tuning language models with custom reward functions.\n\n## When to Use This Skill\n\nUse GRPO training when you need to:\n- **Enforce specific output formats** (e.g., XML tags, JSON, structured reasoning)\n- **Teach verifiable tasks** with objective correctness metrics (math, coding, fact-checking)\n- **Improve reasoning capabilities** by rewarding chain-of-thought patterns\n- **Align models to domain-specific behaviors** without labeled preference data\n- **Optimize for multiple objectives** simultaneously (format + correctness + style)\n\n**Do NOT use GRPO for:**\n- Simple supervised fine-tuning tasks (use SFT instead)\n- Tasks without clear reward signals\n- When you already have high-quality preference pairs (use DPO\u002FPPO instead)\n\n---\n\n## Core Concepts\n\n### 1. GRPO Algorithm Fundamentals\n\n**Key Mechanism:**\n- Generates **multiple completions** for each prompt (group size: 4-16)\n- Compares completions within each group using reward functions\n- Updates policy to favor higher-rewarded responses relative to the group\n\n**Critical Difference from PPO:**\n- No separate reward model needed\n- More sample-efficient (learns from within-group comparisons)\n- Simpler to implement and debug\n\n**Mathematical Intuition:**\n```\nFor each prompt p:\n  1. Generate N completions: {c₁, c₂, ..., cₙ}\n  2. Compute rewards: {r₁, r₂, ..., rₙ}\n  3. Learn to increase probability of high-reward completions\n     relative to low-reward ones in the same group\n```\n\n### 2. Reward Function Design Philosophy\n\n**Golden Rules:**\n1. **Compose multiple reward functions** - Each handles one aspect (format, correctness, style)\n2. **Scale rewards appropriately** - Higher weight = stronger signal\n3. **Use incremental rewards** - Partial credit for partial compliance\n4. **Test rewards independently** - Debug each reward function in isolation\n\n**Reward Function Types:**\n\n| Type | Use Case | Example Weight |\n|------|----------|----------------|\n| **Correctness** | Verifiable tasks (math, code) | 2.0 (highest) |\n| **Format** | Strict structure enforcement | 0.5-1.0 |\n| **Length** | Encourage verbosity\u002Fconciseness | 0.1-0.5 |\n| **Style** | Penalize unwanted patterns | -0.5 to 0.5 |\n\n---\n\n## Implementation Workflow\n\n### Step 1: Dataset Preparation\n\n**Critical Requirements:**\n- Prompts in chat format (list of dicts with 'role' and 'content')\n- Include system prompts to set expectations\n- For verifiable tasks, include ground truth answers as additional columns\n\n**Example Structure:**\n```python\nfrom datasets import load_dataset, Dataset\n\nSYSTEM_PROMPT = \"\"\"\nRespond in the following format:\n\u003Creasoning>\n[Your step-by-step thinking]\n\u003C\u002Freasoning>\n\u003Canswer>\n[Final answer]\n\u003C\u002Fanswer>\n\"\"\"\n\ndef prepare_dataset(raw_data):\n    \"\"\"\n    Transform raw data into GRPO-compatible format.\n\n    Returns: Dataset with columns:\n    - 'prompt': List[Dict] with role\u002Fcontent (system + user messages)\n    - 'answer': str (ground truth, optional but recommended)\n    \"\"\"\n    return raw_data.map(lambda x: {\n        'prompt': [\n            {'role': 'system', 'content': SYSTEM_PROMPT},\n            {'role': 'user', 'content': x['question']}\n        ],\n        'answer': extract_answer(x['raw_answer'])\n    })\n```\n\n**Pro Tips:**\n- Use one-shot or few-shot examples in system prompt for complex formats\n- Keep prompts concise (max_prompt_length: 256-512 tokens)\n- Validate data quality before training (garbage in = garbage out)\n\n### Step 2: Reward Function Implementation\n\n**Template Structure:**\n```python\ndef reward_function_name(\n    prompts,        # List[List[Dict]]: Original prompts\n    completions,    # List[List[Dict]]: Model generations\n    answer=None,    # Optional: Ground truth from dataset\n    **kwargs        # Additional dataset columns\n) -> list[float]:\n    \"\"\"\n    Evaluate completions and return rewards.\n\n    Returns: List of floats (one per completion)\n    \"\"\"\n    # Extract completion text\n    responses = [comp[0]['content'] for comp in completions]\n\n    # Compute rewards\n    rewards = []\n    for response in responses:\n        score = compute_score(response)\n        rewards.append(score)\n\n    return rewards\n```\n\n**Example 1: Correctness Reward (Math\u002FCoding)**\n```python\ndef correctness_reward(prompts, completions, answer, **kwargs):\n    \"\"\"Reward correct answers with high score.\"\"\"\n    responses = [comp[0]['content'] for comp in completions]\n    extracted = [extract_final_answer(r) for r in responses]\n    return [2.0 if ans == gt else 0.0\n            for ans, gt in zip(extracted, answer)]\n```\n\n**Example 2: Format Reward (Structured Output)**\n```python\nimport re\n\ndef format_reward(completions, **kwargs):\n    \"\"\"Reward XML-like structured format.\"\"\"\n    pattern = r'\u003Creasoning>.*?\u003C\u002Freasoning>\\s*\u003Canswer>.*?\u003C\u002Fanswer>'\n    responses = [comp[0]['content'] for comp in completions]\n    return [1.0 if re.search(pattern, r, re.DOTALL) else 0.0\n            for r in responses]\n```\n\n**Example 3: Incremental Format Reward (Partial Credit)**\n```python\ndef incremental_format_reward(completions, **kwargs):\n    \"\"\"Award partial credit for format compliance.\"\"\"\n    responses = [comp[0]['content'] for comp in completions]\n    rewards = []\n\n    for r in responses:\n        score = 0.0\n        if '\u003Creasoning>' in r:\n            score += 0.25\n        if '\u003C\u002Freasoning>' in r:\n            score += 0.25\n        if '\u003Canswer>' in r:\n            score += 0.25\n        if '\u003C\u002Fanswer>' in r:\n            score += 0.25\n        # Penalize extra text after closing tag\n        if r.count('\u003C\u002Fanswer>') == 1:\n            extra_text = r.split('\u003C\u002Fanswer>')[-1].strip()\n            score -= len(extra_text) * 0.001\n        rewards.append(score)\n\n    return rewards\n```\n\n**Critical Insight:**\nCombine 3-5 reward functions for robust training. Order matters less than diversity of signals.\n\n### Step 3: Training Configuration\n\n**Memory-Optimized Config (Small GPU)**\n```python\nfrom trl import GRPOConfig\n\ntraining_args = GRPOConfig(\n    output_dir=\"outputs\u002Fgrpo-model\",\n\n    # Learning rate\n    learning_rate=5e-6,          # Lower = more stable\n    adam_beta1=0.9,\n    adam_beta2=0.99,\n    weight_decay=0.1,\n    warmup_ratio=0.1,\n    lr_scheduler_type='cosine',\n\n    # Batch settings\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=4,  # Effective batch = 4\n\n    # GRPO-specific\n    num_generations=8,            # Group size: 8-16 recommended\n    max_prompt_length=256,\n    max_completion_length=512,\n\n    # Training duration\n    num_train_epochs=1,\n    max_steps=None,               # Or set fixed steps (e.g., 500)\n\n    # Optimization\n    bf16=True,                    # Faster on A100\u002FH100\n    optim=\"adamw_8bit\",          # Memory-efficient optimizer\n    max_grad_norm=0.1,\n\n    # Logging\n    logging_steps=1,\n    save_steps=100,\n    report_to=\"wandb\",            # Or \"none\" for no logging\n)\n```\n\n**High-Performance Config (Large GPU)**\n```python\ntraining_args = GRPOConfig(\n    output_dir=\"outputs\u002Fgrpo-model\",\n    learning_rate=1e-5,\n    per_device_train_batch_size=4,\n    gradient_accumulation_steps=2,\n    num_generations=16,           # Larger groups = better signal\n    max_prompt_length=512,\n    max_completion_length=1024,\n    num_train_epochs=1,\n    bf16=True,\n    use_vllm=True,                # Fast generation with vLLM\n    logging_steps=10,\n)\n```\n\n**Critical Hyperparameters:**\n\n| Parameter | Impact | Tuning Advice |\n|-----------|--------|---------------|\n| `num_generations` | Group size for comparison | Start with 8, increase to 16 if GPU allows |\n| `learning_rate` | Convergence speed\u002Fstability | 5e-6 (safe), 1e-5 (faster, riskier) |\n| `max_completion_length` | Output verbosity | Match your task (512 for reasoning, 256 for short answers) |\n| `gradient_accumulation_steps` | Effective batch size | Increase if GPU memory limited |\n\n### Step 4: Model Setup and Training\n\n**Standard Setup (Transformers)**\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom peft import LoraConfig\nfrom trl import GRPOTrainer\n\n# Load model\nmodel_name = \"Qwen\u002FQwen2.5-1.5B-Instruct\"\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    torch_dtype=torch.bfloat16,\n    attn_implementation=\"flash_attention_2\",  # 2-3x faster\n    device_map=\"auto\"\n)\n\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# Optional: LoRA for parameter-efficient training\npeft_config = LoraConfig(\n    r=16,                         # Rank (higher = more capacity)\n    lora_alpha=32,               # Scaling factor (typically 2*r)\n    target_modules=[\n        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n        \"gate_proj\", \"up_proj\", \"down_proj\"\n    ],\n    task_type=\"CAUSAL_LM\",\n    lora_dropout=0.05,\n)\n\n# Initialize trainer\ntrainer = GRPOTrainer(\n    model=model,\n    processing_class=tokenizer,\n    reward_funcs=[\n        incremental_format_reward,\n        format_reward,\n        correctness_reward,\n    ],\n    args=training_args,\n    train_dataset=dataset,\n    peft_config=peft_config,      # Remove for full fine-tuning\n)\n\n# Train\ntrainer.train()\n\n# Save\ntrainer.save_model(\"final_model\")\n```\n\n**Unsloth Setup (2-3x Faster)**\n```python\nfrom unsloth import FastLanguageModel\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"google\u002Fgemma-3-1b-it\",\n    max_seq_length=1024,\n    load_in_4bit=True,\n    fast_inference=True,\n    max_lora_rank=32,\n)\n\nmodel = FastLanguageModel.get_peft_model(\n    model,\n    r=32,\n    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n                    \"gate_proj\", \"up_proj\", \"down_proj\"],\n    lora_alpha=32,\n    use_gradient_checkpointing=\"unsloth\",\n)\n\n# Rest is identical to standard setup\ntrainer = GRPOTrainer(model=model, ...)\ntrainer.train()\n```\n\n---\n\n## Critical Training Insights\n\n### 1. Loss Behavior (EXPECTED PATTERN)\n- **Loss starts near 0 and INCREASES during training**\n- This is CORRECT - loss measures KL divergence from initial policy\n- Model is learning (diverging from original behavior to optimize rewards)\n- Monitor reward metrics instead of loss for progress\n\n### 2. Reward Tracking\nKey metrics to watch:\n- `reward`: Average across all completions\n- `reward_std`: Diversity within groups (should remain > 0)\n- `kl`: KL divergence from reference (should grow moderately)\n\n**Healthy Training Pattern:**\n```\nStep   Reward    Reward_Std   KL\n100    0.5       0.3          0.02\n200    0.8       0.25         0.05\n300    1.2       0.2          0.08  ← Good progression\n400    1.5       0.15         0.12\n```\n\n**Warning Signs:**\n- Reward std → 0 (model collapsing to single response)\n- KL exploding (> 0.5) (diverging too much, reduce LR)\n- Reward stuck (reward functions too harsh or model capacity issue)\n\n### 3. Common Pitfalls and Solutions\n\n| Problem | Symptom | Solution |\n|---------|---------|----------|\n| **Mode collapse** | All completions identical | Increase `num_generations`, add diversity penalty |\n| **No learning** | Flat rewards | Check reward function logic, increase LR |\n| **OOM errors** | GPU memory exceeded | Reduce `num_generations`, enable gradient checkpointing |\n| **Slow training** | \u003C 1 it\u002Fs | Enable `use_vllm=True`, use Unsloth, reduce seq length |\n| **Format ignored** | Model doesn't follow structure | Increase format reward weight, add incremental rewards |\n\n---\n\n## Advanced Patterns\n\n### 1. Multi-Stage Training\nFor complex tasks, train in stages:\n\n```python\n# Stage 1: Format compliance (epochs=1)\ntrainer_stage1 = GRPOTrainer(\n    model=model,\n    reward_funcs=[incremental_format_reward, format_reward],\n    ...\n)\ntrainer_stage1.train()\n\n# Stage 2: Correctness (epochs=1)\ntrainer_stage2 = GRPOTrainer(\n    model=model,\n    reward_funcs=[format_reward, correctness_reward],\n    ...\n)\ntrainer_stage2.train()\n```\n\n### 2. Adaptive Reward Scaling\n```python\nclass AdaptiveReward:\n    def __init__(self, base_reward_func, initial_weight=1.0):\n        self.func = base_reward_func\n        self.weight = initial_weight\n\n    def __call__(self, *args, **kwargs):\n        rewards = self.func(*args, **kwargs)\n        return [r * self.weight for r in rewards]\n\n    def adjust_weight(self, success_rate):\n        \"\"\"Increase weight if model struggling, decrease if succeeding.\"\"\"\n        if success_rate \u003C 0.3:\n            self.weight *= 1.2\n        elif success_rate > 0.8:\n            self.weight *= 0.9\n```\n\n### 3. Custom Dataset Integration\n```python\ndef load_custom_knowledge_base(csv_path):\n    \"\"\"Example: School communication platform docs.\"\"\"\n    import pandas as pd\n    df = pd.read_csv(csv_path)\n\n    dataset = Dataset.from_pandas(df).map(lambda x: {\n        'prompt': [\n            {'role': 'system', 'content': CUSTOM_SYSTEM_PROMPT},\n            {'role': 'user', 'content': x['question']}\n        ],\n        'answer': x['expert_answer']\n    })\n    return dataset\n```\n\n---\n\n## Deployment and Inference\n\n### Save and Merge LoRA\n```python\n# Merge LoRA adapters into base model\nif hasattr(trainer.model, 'merge_and_unload'):\n    merged_model = trainer.model.merge_and_unload()\n    merged_model.save_pretrained(\"production_model\")\n    tokenizer.save_pretrained(\"production_model\")\n```\n\n### Inference Example\n```python\nfrom transformers import pipeline\n\ngenerator = pipeline(\n    \"text-generation\",\n    model=\"production_model\",\n    tokenizer=tokenizer\n)\n\nresult = generator(\n    [\n        {'role': 'system', 'content': SYSTEM_PROMPT},\n        {'role': 'user', 'content': \"What is 15 + 27?\"}\n    ],\n    max_new_tokens=256,\n    do_sample=True,\n    temperature=0.7,\n    top_p=0.9\n)\nprint(result[0]['generated_text'])\n```\n\n---\n\n## Best Practices Checklist\n\n**Before Training:**\n- [ ] Validate dataset format (prompts as List[Dict])\n- [ ] Test reward functions on sample data\n- [ ] Calculate expected max_prompt_length from data\n- [ ] Choose appropriate num_generations based on GPU memory\n- [ ] Set up logging (wandb recommended)\n\n**During Training:**\n- [ ] Monitor reward progression (should increase)\n- [ ] Check reward_std (should stay > 0.1)\n- [ ] Watch for OOM errors (reduce batch size if needed)\n- [ ] Sample generations every 50-100 steps\n- [ ] Validate format compliance on holdout set\n\n**After Training:**\n- [ ] Merge LoRA weights if using PEFT\n- [ ] Test on diverse prompts\n- [ ] Compare to baseline model\n- [ ] Document reward weights and hyperparameters\n- [ ] Save reproducibility config\n\n---\n\n## Troubleshooting Guide\n\n### Debugging Workflow\n1. **Isolate reward functions** - Test each independently\n2. **Check data distribution** - Ensure diversity in prompts\n3. **Reduce complexity** - Start with single reward, add gradually\n4. **Monitor generations** - Print samples every N steps\n5. **Validate extraction logic** - Ensure answer parsing works\n\n### Quick Fixes\n```python\n# Debug reward function\ndef debug_reward(completions, **kwargs):\n    responses = [comp[0]['content'] for comp in completions]\n    for i, r in enumerate(responses[:2]):  # Print first 2\n        print(f\"Response {i}: {r[:200]}...\")\n    return [1.0] * len(responses)  # Dummy rewards\n\n# Test without training\ntrainer = GRPOTrainer(..., reward_funcs=[debug_reward])\ntrainer.generate_completions(dataset[:1])  # Generate without updating\n```\n\n---\n\n## References and Resources\n\n**Official Documentation:**\n- TRL GRPO Trainer: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Ftrl\u002Fgrpo_trainer\n- DeepSeek R1 Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.12948\n- Unsloth Docs: https:\u002F\u002Fdocs.unsloth.ai\u002F\n\n**Example Repositories:**\n- Open R1 Implementation: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fopen-r1\n- TRL Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl\u002Ftree\u002Fmain\u002Fexamples\n\n**Recommended Reading:**\n- Progressive Disclosure Pattern for agent instructions\n- Reward shaping in RL (Ng et al.)\n- LoRA paper (Hu et al., 2021)\n\n---\n\n## Usage Instructions for Agents\n\nWhen this skill is loaded:\n\n1. **Read this entire file** before implementing GRPO training\n2. **Start with the simplest reward function** (e.g., length-based) to validate setup\n3. **Use the templates** in `templates\u002F` directory as starting points\n4. **Reference examples** in `examples\u002F` for task-specific implementations\n5. **Follow the workflow** sequentially (don't skip steps)\n6. **Debug incrementally** - add one reward function at a time\n\n**Critical Reminders:**\n- Always use multiple reward functions (3-5 is optimal)\n- Monitor reward metrics, not loss\n- Test reward functions before training\n- Start small (num_generations=4), scale up gradually\n- Save checkpoints frequently (every 100 steps)\n\nThis skill is designed for **expert-level implementation**. Beginners should start with supervised fine-tuning before attempting GRPO.\n\n\n\n","# GRPO\u002FRL Training with TRL\n\nExpert-level guidance for implementing Group Relative Policy Optimization (GRPO) using the Transformer Reinforcement Learning (TRL) library. This skill provides battle-tested patterns, critical insights, and production-ready workflows for fine-tuning language models with custom reward functions.\n\n## When to Use This Skill\n\nUse GRPO training when you need to:\n- **Enforce specific output formats** (e.g., XML tags, JSON, structured reasoning)\n- **Teach verifiable tasks** with objective correctness metrics (math, coding, fact-checking)\n- **Improve reasoning capabilities** by rewarding chain-of-thought patterns\n- **Align models to domain-specific behaviors** without labeled preference data\n- **Optimize for multiple objectives** simultaneously (format + correctness + style)\n\n**Do NOT use GRPO for:**\n- Simple supervised fine-tuning tasks (use SFT instead)\n- Tasks without clear reward signals\n- When you already have high-quality preference pairs (use DPO\u002FPPO instead)\n\n---\n\n## Core Concepts\n\n### 1. GRPO Algorithm Fundamentals\n\n**Key Mechanism:**\n- Generates **multiple completions** for each prompt (group size: 4-16)\n- Compares completions within each group using reward functions\n- Updates policy to favor higher-rewarded responses relative to the group\n\n**Critical Difference from PPO:**\n- No separate reward model needed\n- More sample-efficient (learns from within-group comparisons)\n- Simpler to implement and debug\n\n**Mathematical Intuition:**\n```\nFor each prompt p:\n  1. Generate N completions: {c₁, c₂, ..., cₙ}\n  2. Compute rewards: {r₁, r₂, ..., rₙ}\n  3. Learn to increase probability of high-reward completions\n     relative to low-reward ones in the same group\n```\n\n### 2. Reward Function Design Philosophy\n\n**Golden Rules:**\n1. **Compose multiple reward functions** - Each handles one aspect (format, correctness, style)\n2. **Scale rewards appropriately** - Higher weight = stronger signal\n3. **Use incremental rewards** - Partial credit for partial compliance\n4. **Test rewards independently** - Debug each reward function in isolation\n\n**Reward Function Types:**\n\n| Type | Use Case | Example Weight |\n|------|----------|----------------|\n| **Correctness** | Verifiable tasks (math, code) | 2.0 (highest) |\n| **Format** | Strict structure enforcement | 0.5-1.0 |\n| **Length** | Encourage verbosity\u002Fconciseness | 0.1-0.5 |\n| **Style** | Penalize unwanted patterns | -0.5 to 0.5 |\n\n---\n\n## Implementation Workflow\n\n### Step 1: Dataset Preparation\n\n**Critical Requirements:**\n- Prompts in chat format (list of dicts with 'role' and 'content')\n- Include system prompts to set expectations\n- For verifiable tasks, include ground truth answers as additional columns\n\n**Example Structure:**\n```python\nfrom datasets import load_dataset, Dataset\n\nSYSTEM_PROMPT = \"\"\"\nRespond in the following format:\n\u003Creasoning>\n[Your step-by-step thinking]\n\u003C\u002Freasoning>\n\u003Canswer>\n[Final answer]\n\u003C\u002Fanswer>\n\"\"\"\n\ndef prepare_dataset(raw_data):\n    \"\"\"\n    Transform raw data into GRPO-compatible format.\n\n    Returns: Dataset with columns:\n    - 'prompt': List[Dict] with role\u002Fcontent (system + user messages)\n    - 'answer': str (ground truth, optional but recommended)\n    \"\"\"\n    return raw_data.map(lambda x: {\n        'prompt': [\n            {'role': 'system', 'content': SYSTEM_PROMPT},\n            {'role': 'user', 'content': x['question']}\n        ],\n        'answer': extract_answer(x['raw_answer'])\n    })\n```\n\n**Pro Tips:**\n- Use one-shot or few-shot examples in system prompt for complex formats\n- Keep prompts concise (max_prompt_length: 256-512 tokens)\n- Validate data quality before training (garbage in = garbage out)\n\n### Step 2: Reward Function Implementation\n\n**Template Structure:**\n```python\ndef reward_function_name(\n    prompts,        # List[List[Dict]]: Original prompts\n    completions,    # List[List[Dict]]: Model generations\n    answer=None,    # Optional: Ground truth from dataset\n    **kwargs        # Additional dataset columns\n) -> list[float]:\n    \"\"\"\n    Evaluate completions and return rewards.\n\n    Returns: List of floats (one per completion)\n    \"\"\"\n    # Extract completion text\n    responses = [comp[0]['content'] for comp in completions]\n\n    # Compute rewards\n    rewards = []\n    for response in responses:\n        score = compute_score(response)\n        rewards.append(score)\n\n    return rewards\n```\n\n**Example 1: Correctness Reward (Math\u002FCoding)**\n```python\ndef correctness_reward(prompts, completions, answer, **kwargs):\n    \"\"\"Reward correct answers with high score.\"\"\"\n    responses = [comp[0]['content'] for comp in completions]\n    extracted = [extract_final_answer(r) for r in responses]\n    return [2.0 if ans == gt else 0.0\n            for ans, gt in zip(extracted, answer)]\n```\n\n**Example 2: Format Reward (Structured Output)**\n```python\nimport re\n\ndef format_reward(completions, **kwargs):\n    \"\"\"Reward XML-like structured format.\"\"\"\n    pattern = r'\u003Creasoning>.*?\u003C\u002Freasoning>\\s*\u003Canswer>.*?\u003C\u002Fanswer>'\n    responses = [comp[0]['content'] for comp in completions]\n    return [1.0 if re.search(pattern, r, re.DOTALL) else 0.0\n            for r in responses]\n```\n\n**Example 3: Incremental Format Reward (Partial Credit)**\n```python\ndef incremental_format_reward(completions, **kwargs):\n    \"\"\"Award partial credit for format compliance.\"\"\"\n    responses = [comp[0]['content'] for comp in completions]\n    rewards = []\n\n    for r in responses:\n        score = 0.0\n        if '\u003Creasoning>' in r:\n            score += 0.25\n        if '\u003C\u002Freasoning>' in r:\n            score += 0.25\n        if '\u003Canswer>' in r:\n            score += 0.25\n        if '\u003C\u002Fanswer>' in r:\n            score += 0.25\n        # Penalize extra text after closing tag\n        if r.count('\u003C\u002Fanswer>') == 1:\n            extra_text = r.split('\u003C\u002Fanswer>')[-1].strip()\n            score -= len(extra_text) * 0.001\n        rewards.append(score)\n\n    return rewards\n```\n\n**Critical Insight:**\nCombine 3-5 reward functions for robust training. Order matters less than diversity of signals.\n\n### Step 3: Training Configuration\n\n**Memory-Optimized Config (Small GPU)**\n```python\nfrom trl import GRPOConfig\n\ntraining_args = GRPOConfig(\n    output_dir=\"outputs\u002Fgrpo-model\",\n\n    # Learning rate\n    learning_rate=5e-6,          # Lower = more stable\n    adam_beta1=0.9,\n    adam_beta2=0.99,\n    weight_decay=0.1,\n    warmup_ratio=0.1,\n    lr_scheduler_type='cosine',\n\n    # Batch settings\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=4,  # Effective batch = 4\n\n    # GRPO-specific\n    num_generations=8,            # Group size: 8-16 recommended\n    max_prompt_length=256,\n    max_completion_length=512,\n\n    # Training duration\n    num_train_epochs=1,\n    max_steps=None,               # Or set fixed steps (e.g., 500)\n\n    # Optimization\n    bf16=True,                    # Faster on A100\u002FH100\n    optim=\"adamw_8bit\",          # Memory-efficient optimizer\n    max_grad_norm=0.1,\n\n    # Logging\n    logging_steps=1,\n    save_steps=100,\n    report_to=\"wandb\",            # Or \"none\" for no logging\n)\n```\n\n**High-Performance Config (Large GPU)**\n```python\ntraining_args = GRPOConfig(\n    output_dir=\"outputs\u002Fgrpo-model\",\n    learning_rate=1e-5,\n    per_device_train_batch_size=4,\n    gradient_accumulation_steps=2,\n    num_generations=16,           # Larger groups = better signal\n    max_prompt_length=512,\n    max_completion_length=1024,\n    num_train_epochs=1,\n    bf16=True,\n    use_vllm=True,                # Fast generation with vLLM\n    logging_steps=10,\n)\n```\n\n**Critical Hyperparameters:**\n\n| Parameter | Impact | Tuning Advice |\n|-----------|--------|---------------|\n| `num_generations` | Group size for comparison | Start with 8, increase to 16 if GPU allows |\n| `learning_rate` | Convergence speed\u002Fstability | 5e-6 (safe), 1e-5 (faster, riskier) |\n| `max_completion_length` | Output verbosity | Match your task (512 for reasoning, 256 for short answers) |\n| `gradient_accumulation_steps` | Effective batch size | Increase if GPU memory limited |\n\n### Step 4: Model Setup and Training\n\n**Standard Setup (Transformers)**\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom peft import LoraConfig\nfrom trl import GRPOTrainer\n\n# Load model\nmodel_name = \"Qwen\u002FQwen2.5-1.5B-Instruct\"\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    torch_dtype=torch.bfloat16,\n    attn_implementation=\"flash_attention_2\",  # 2-3x faster\n    device_map=\"auto\"\n)\n\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# Optional: LoRA for parameter-efficient training\npeft_config = LoraConfig(\n    r=16,                         # Rank (higher = more capacity)\n    lora_alpha=32,               # Scaling factor (typically 2*r)\n    target_modules=[\n        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n        \"gate_proj\", \"up_proj\", \"down_proj\"\n    ],\n    task_type=\"CAUSAL_LM\",\n    lora_dropout=0.05,\n)\n\n# Initialize trainer\ntrainer = GRPOTrainer(\n    model=model,\n    processing_class=tokenizer,\n    reward_funcs=[\n        incremental_format_reward,\n        format_reward,\n        correctness_reward,\n    ],\n    args=training_args,\n    train_dataset=dataset,\n    peft_config=peft_config,      # Remove for full fine-tuning\n)\n\n# Train\ntrainer.train()\n\n# Save\ntrainer.save_model(\"final_model\")\n```\n\n**Unsloth Setup (2-3x Faster)**\n```python\nfrom unsloth import FastLanguageModel\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"google\u002Fgemma-3-1b-it\",\n    max_seq_length=1024,\n    load_in_4bit=True,\n    fast_inference=True,\n    max_lora_rank=32,\n)\n\nmodel = FastLanguageModel.get_peft_model(\n    model,\n    r=32,\n    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n                    \"gate_proj\", \"up_proj\", \"down_proj\"],\n    lora_alpha=32,\n    use_gradient_checkpointing=\"unsloth\",\n)\n\n# Rest is identical to standard setup\ntrainer = GRPOTrainer(model=model, ...)\ntrainer.train()\n```\n\n---\n\n## Critical Training Insights\n\n### 1. Loss Behavior (EXPECTED PATTERN)\n- **Loss starts near 0 and INCREASES during training**\n- This is CORRECT - loss measures KL divergence from initial policy\n- Model is learning (diverging from original behavior to optimize rewards)\n- Monitor reward metrics instead of loss for progress\n\n### 2. Reward Tracking\nKey metrics to watch:\n- `reward`: Average across all completions\n- `reward_std`: Diversity within groups (should remain > 0)\n- `kl`: KL divergence from reference (should grow moderately)\n\n**Healthy Training Pattern:**\n```\nStep   Reward    Reward_Std   KL\n100    0.5       0.3          0.02\n200    0.8       0.25         0.05\n300    1.2       0.2          0.08  ← Good progression\n400    1.5       0.15         0.12\n```\n\n**Warning Signs:**\n- Reward std → 0 (model collapsing to single response)\n- KL exploding (> 0.5) (diverging too much, reduce LR)\n- Reward stuck (reward functions too harsh or model capacity issue)\n\n### 3. Common Pitfalls and Solutions\n\n| Problem | Symptom | Solution |\n|---------|---------|----------|\n| **Mode collapse** | All completions identical | Increase `num_generations`, add diversity penalty |\n| **No learning** | Flat rewards | Check reward function logic, increase LR |\n| **OOM errors** | GPU memory exceeded | Reduce `num_generations`, enable gradient checkpointing |\n| **Slow training** | \u003C 1 it\u002Fs | Enable `use_vllm=True`, use Unsloth, reduce seq length |\n| **Format ignored** | Model doesn't follow structure | Increase format reward weight, add incremental rewards |\n\n---\n\n## Advanced Patterns\n\n### 1. Multi-Stage Training\nFor complex tasks, train in stages:\n\n```python\n# Stage 1: Format compliance (epochs=1)\ntrainer_stage1 = GRPOTrainer(\n    model=model,\n    reward_funcs=[incremental_format_reward, format_reward],\n    ...\n)\ntrainer_stage1.train()\n\n# Stage 2: Correctness (epochs=1)\ntrainer_stage2 = GRPOTrainer(\n    model=model,\n    reward_funcs=[format_reward, correctness_reward],\n    ...\n)\ntrainer_stage2.train()\n```\n\n### 2. Adaptive Reward Scaling\n```python\nclass AdaptiveReward:\n    def __init__(self, base_reward_func, initial_weight=1.0):\n        self.func = base_reward_func\n        self.weight = initial_weight\n\n    def __call__(self, *args, **kwargs):\n        rewards = self.func(*args, **kwargs)\n        return [r * self.weight for r in rewards]\n\n    def adjust_weight(self, success_rate):\n        \"\"\"Increase weight if model struggling, decrease if succeeding.\"\"\"\n        if success_rate \u003C 0.3:\n            self.weight *= 1.2\n        elif success_rate > 0.8:\n            self.weight *= 0.9\n```\n\n### 3. Custom Dataset Integration\n```python\ndef load_custom_knowledge_base(csv_path):\n    \"\"\"Example: School communication platform docs.\"\"\"\n    import pandas as pd\n    df = pd.read_csv(csv_path)\n\n    dataset = Dataset.from_pandas(df).map(lambda x: {\n        'prompt': [\n            {'role': 'system', 'content': CUSTOM_SYSTEM_PROMPT},\n            {'role': 'user', 'content': x['question']}\n        ],\n        'answer': x['expert_answer']\n    })\n    return dataset\n```\n\n---\n\n## Deployment and Inference\n\n### Save and Merge LoRA\n```python\n# Merge LoRA adapters into base model\nif hasattr(trainer.model, 'merge_and_unload'):\n    merged_model = trainer.model.merge_and_unload()\n    merged_model.save_pretrained(\"production_model\")\n    tokenizer.save_pretrained(\"production_model\")\n```\n\n### Inference Example\n```python\nfrom transformers import pipeline\n\ngenerator = pipeline(\n    \"text-generation\",\n    model=\"production_model\",\n    tokenizer=tokenizer\n)\n\nresult = generator(\n    [\n        {'role': 'system', 'content': SYSTEM_PROMPT},\n        {'role': 'user', 'content': \"What is 15 + 27?\"}\n    ],\n    max_new_tokens=256,\n    do_sample=True,\n    temperature=0.7,\n    top_p=0.9\n)\nprint(result[0]['generated_text'])\n```\n\n---\n\n## Best Practices Checklist\n\n**Before Training:**\n- [ ] Validate dataset format (prompts as List[Dict])\n- [ ] Test reward functions on sample data\n- [ ] Calculate expected max_prompt_length from data\n- [ ] Choose appropriate num_generations based on GPU memory\n- [ ] Set up logging (wandb recommended)\n\n**During Training:**\n- [ ] Monitor reward progression (should increase)\n- [ ] Check reward_std (should stay > 0.1)\n- [ ] Watch for OOM errors (reduce batch size if needed)\n- [ ] Sample generations every 50-100 steps\n- [ ] Validate format compliance on holdout set\n\n**After Training:**\n- [ ] Merge LoRA weights if using PEFT\n- [ ] Test on diverse prompts\n- [ ] Compare to baseline model\n- [ ] Document reward weights and hyperparameters\n- [ ] Save reproducibility config\n\n---\n\n## Troubleshooting Guide\n\n### Debugging Workflow\n1. **Isolate reward functions** - Test each independently\n2. **Check data distribution** - Ensure diversity in prompts\n3. **Reduce complexity** - Start with single reward, add gradually\n4. **Monitor generations** - Print samples every N steps\n5. **Validate extraction logic** - Ensure answer parsing works\n\n### Quick Fixes\n```python\n# Debug reward function\ndef debug_reward(completions, **kwargs):\n    responses = [comp[0]['content'] for comp in completions]\n    for i, r in enumerate(responses[:2]):  # Print first 2\n        print(f\"Response {i}: {r[:200]}...\")\n    return [1.0] * len(responses)  # Dummy rewards\n\n# Test without training\ntrainer = GRPOTrainer(..., reward_funcs=[debug_reward])\ntrainer.generate_completions(dataset[:1])  # Generate without updating\n```\n\n---\n\n## References and Resources\n\n**Official Documentation:**\n- TRL GRPO Trainer: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Ftrl\u002Fgrpo_trainer\n- DeepSeek R1 Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.12948\n- Unsloth Docs: https:\u002F\u002Fdocs.unsloth.ai\u002F\n\n**Example Repositories:**\n- Open R1 Implementation: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fopen-r1\n- TRL Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl\u002Ftree\u002Fmain\u002Fexamples\n\n**Recommended Reading:**\n- Progressive Disclosure Pattern for agent instructions\n- Reward shaping in RL (Ng et al.)\n- LoRA paper (Hu et al., 2021)\n\n---\n\n## Usage Instructions for Agents\n\nWhen this skill is loaded:\n\n1. **Read this entire file** before implementing GRPO training\n2. **Start with the simplest reward function** (e.g., length-based) to validate setup\n3. **Use the templates** in `templates\u002F` directory as starting points\n4. **Reference examples** in `examples\u002F` for task-specific implementations\n5. **Follow the workflow** sequentially (don't skip steps)\n6. **Debug incrementally** - add one reward function at a time\n\n**Critical Reminders:**\n- Always use multiple reward functions (3-5 is optimal)\n- Monitor reward metrics, not loss\n- Test reward functions before training\n- Start small (num_generations=4), scale up gradually\n- Save checkpoints frequently (every 100 steps)\n\nThis skill is designed for **expert-level implementation**. Beginners should start with supervised fine-tuning before attempting GRPO.\n\n\n\n",{"name":5,"tags":23,"author":34,"license":14,"version":35,"description":7,"dependencies":36},[24,25,26,27,28,29,30,31,32,33],"Post-Training","Reinforcement Learning","GRPO","TRL","RLHF","Reward Modeling","Reasoning","DPO","PPO","Structured Output","Orchestra Research","1.0.0",[37,38,39,40,41],"transformers>=4.47.0","trl>=0.14.0","datasets>=3.2.0","peft>=0.14.0","torch","main","d046d088ce5a79a083f0abc0fe1b7ded73bc5b1f","claude-config\u002Fskills\u002Fgrpo-rl-training","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent","https:\u002F\u002Fgithub.com\u002FSansar35","2026-05-10 16:04:53",{"source":8,"skillName":5,"commands":49,"prompt":59,"zipAvailable":60},[50,53,56],{"tool":51,"command":52},"npx","npx skills add Sansar35\u002Fclaude-agent --skill grpo-rl-training",{"tool":54,"command":55},"bunx","bunx skills add Sansar35\u002Fclaude-agent --skill grpo-rl-training",{"tool":57,"command":58},"pnpm","pnpm dlx skills add Sansar35\u002Fclaude-agent --skill grpo-rl-training","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fgrpo-rl-training\u002FSKILL.md",true,[62,68,74,78],{"path":63,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":66,"sourceHtmlUrl":67},".skill-source.json","REFERENCE","参考文档",342,"https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fgrpo-rl-training\u002F.skill-source.json",{"path":69,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":72,"sourceHtmlUrl":73},"examples\u002Freward_functions_library.py","SCRIPT","脚本",11568,"https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fgrpo-rl-training\u002Fexamples\u002Freward_functions_library.py",{"path":75,"resourceType":64,"resourceTypeDesc":65,"sizeBytes":76,"sourceHtmlUrl":77},"README.md",3514,"https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fgrpo-rl-training\u002FREADME.md",{"path":79,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":80,"sourceHtmlUrl":81},"templates\u002Fbasic_grpo_training.py",6122,"https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fgrpo-rl-training\u002Ftemplates\u002Fbasic_grpo_training.py",[83,90,97,104,111,119],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":18},"01M72DVAN1JYEA302MT0C5MGD6","zustand-store-ts","Create Zustand stores following established patterns with proper TypeScript types and middleware.","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fzustand-store-ts\u002FSKILL.md",[],[],{"skillNo":91,"name":92,"nameZh":6,"description":93,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":94,"categories":95,"keywords":96,"publishedAt":18},"01M72DV8J5NBRRM4RPDFRV95N7","zod-validation-expert","Expert in Zod — TypeScript-first schema validation. Covers parsing, custom errors, refinements, type inference, and integration with React Hook Form, Next.js, and tRPC.","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fzod-validation-expert\u002FSKILL.md",[],[],{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":103,"publishedAt":18},"01M72DV5EMHW0BHP9BZ167MDCB","zinc-database","Access ZINC (230M+ purchasable compounds). Search by ZINC ID\u002FSMILES, similarity searches, 3D-ready structures for docking, analog discovery, for virtual screening and drug discovery.","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fzinc-database\u002FSKILL.md",[],[],{"skillNo":105,"name":106,"nameZh":6,"description":107,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":108,"categories":109,"keywords":110,"publishedAt":18},"01M72DV216QY0E693ZPEVQ0DBS","zarr-python","Chunked N-D arrays for cloud storage. Compressed arrays, parallel I\u002FO, S3\u002FGCS integration, NumPy\u002FDask\u002FXarray compatible, for large-scale scientific computing pipelines.","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fzarr-python\u002FSKILL.md",[],[],{"skillNo":112,"name":113,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":115,"sourceHtmlUrl":116,"categories":117,"keywords":118,"publishedAt":18},"01M72DTXQREMCRKA8NHCVKHN4W","zapier-workflows","Manage and trigger pre-built Zapier workflows and MCP tool orchestration. Use when user mentions workflows, Zaps, automations, daily digest, research, search, lead tracking, expenses, or asks to \"run\" any process. Also handles Perplexity-based research and Google Sheets data tracking.","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fzapier-workflows\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fzapier-workflows\u002FSKILL.md",[],[],{"skillNo":120,"name":121,"nameZh":6,"description":122,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":123,"categories":124,"keywords":125,"publishedAt":18},"01M72DTT17TBSJ0Y8J2XZ4C9TZ","zapier-make-patterns","No-code automation democratizes workflow building. Zapier and Make (formerly Integromat) let non-developers automate business processes without writing code. But no-code doesn't mean no-complexity - these platforms have their own patterns, pitfalls, and breaking points.  This skill covers when to use which platform, how to build reliable automations, and when to graduate to code-based solutions. Key insight: Zapier optimizes for simplicity and integrations (7000+ apps), Make optimizes for power","https:\u002F\u002Fgithub.com\u002FSansar35\u002Fclaude-agent\u002Fblob\u002Fmain\u002Fclaude-config\u002Fskills\u002Fzapier-make-patterns\u002FSKILL.md",[],[]]