[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6CAPFRH003K1SPX4QC47GJB":3,"skill-same:01M6CAPFRH003K1SPX4QC47GJB":90},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":59,"commitSha":60,"skillDirectory":61,"forksCount":62,"repositoryHtmlUrl":63,"repositoryHomepageUrl":6,"ownerHtmlUrl":64,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":65,"install":66,"resources":79},"01M6CAPFRH003K1SPX4QC47GJB","peft-fine-tuning",null,"Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train \u003C1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.","adnichols\u002Fai-configs","adnichols","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F493746?v=4","01M6C33VRY9WF6XWYPT1AFA7JJ","Aaron Nichols","01M6C33VWTNKGRM92E9G643V4W",5,"MIT","https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fprofiles\u002Fwriting\u002Fskills\u002Fmlops\u002Ftraining\u002Fpeft\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37],"peft fine tuning","fine tuning","tuning","QLoRA","models","minimal","use","Transformers","Large Models","HuggingFace","llms","LoRA","Fine-Tuning","Optimization","library","gpu","official","research","memory","2026-11-10 10:12:30",false,"---\nname: peft-fine-tuning\ndescription: Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train \u003C1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ndependencies: [peft>=0.13.0, transformers>=4.45.0, torch>=2.0.0, bitsandbytes>=0.43.0]\nmetadata:\n  hermes:\n    tags: [Fine-Tuning, PEFT, LoRA, QLoRA, Parameter-Efficient, Adapters, Low-Rank, Memory Optimization, Multi-Adapter]\n\n---\n\n# PEFT (Parameter-Efficient Fine-Tuning)\n\nFine-tune LLMs by training \u003C1% of parameters using LoRA, QLoRA, and 25+ adapter methods.\n\n## When to use PEFT\n\n**Use PEFT\u002FLoRA when:**\n- Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100)\n- Need to train \u003C1% parameters (6MB adapters vs 14GB full model)\n- Want fast iteration with multiple task-specific adapters\n- Deploying multiple fine-tuned variants from one base model\n\n**Use QLoRA (PEFT + quantization) when:**\n- Fine-tuning 70B models on single 24GB GPU\n- Memory is the primary constraint\n- Can accept ~5% quality trade-off vs full fine-tuning\n\n**Use full fine-tuning instead when:**\n- Training small models (\u003C1B parameters)\n- Need maximum quality and have compute budget\n- Significant domain shift requires updating all weights\n\n## Quick start\n\n### Installation\n\n```bash\n# Basic installation\npip install peft\n\n# With quantization support (recommended)\npip install peft bitsandbytes\n\n# Full stack\npip install peft transformers accelerate bitsandbytes datasets\n```\n\n### LoRA fine-tuning (standard)\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer\nfrom peft import get_peft_model, LoraConfig, TaskType\nfrom datasets import load_dataset\n\n# Load base model\nmodel_name = \"meta-llama\u002FLlama-3.1-8B\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=\"auto\", device_map=\"auto\")\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# LoRA configuration\nlora_config = LoraConfig(\n    task_type=TaskType.CAUSAL_LM,\n    r=16,                          # Rank (8-64, higher = more capacity)\n    lora_alpha=32,                 # Scaling factor (typically 2*r)\n    lora_dropout=0.05,             # Dropout for regularization\n    target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\"],  # Attention layers\n    bias=\"none\"                    # Don't train biases\n)\n\n# Apply LoRA\nmodel = get_peft_model(model, lora_config)\nmodel.print_trainable_parameters()\n# Output: trainable params: 13,631,488 || all params: 8,043,307,008 || trainable%: 0.17%\n\n# Prepare dataset\ndataset = load_dataset(\"databricks\u002Fdatabricks-dolly-15k\", split=\"train\")\n\ndef tokenize(example):\n    text = f\"### Instruction:\\n{example['instruction']}\\n\\n### Response:\\n{example['response']}\"\n    return tokenizer(text, truncation=True, max_length=512, padding=\"max_length\")\n\ntokenized = dataset.map(tokenize, remove_columns=dataset.column_names)\n\n# Training\ntraining_args = TrainingArguments(\n    output_dir=\".\u002Flora-llama\",\n    num_train_epochs=3,\n    per_device_train_batch_size=4,\n    gradient_accumulation_steps=4,\n    learning_rate=2e-4,\n    fp16=True,\n    logging_steps=10,\n    save_strategy=\"epoch\"\n)\n\ntrainer = Trainer(\n    model=model,\n    args=training_args,\n    train_dataset=tokenized,\n    data_collator=lambda data: {\"input_ids\": torch.stack([f[\"input_ids\"] for f in data]),\n                                 \"attention_mask\": torch.stack([f[\"attention_mask\"] for f in data]),\n                                 \"labels\": torch.stack([f[\"input_ids\"] for f in data])}\n)\n\ntrainer.train()\n\n# Save adapter only (6MB vs 16GB)\nmodel.save_pretrained(\".\u002Flora-llama-adapter\")\n```\n\n### QLoRA fine-tuning (memory-efficient)\n\n```python\nfrom transformers import AutoModelForCausalLM, BitsAndBytesConfig\nfrom peft import get_peft_model, LoraConfig, prepare_model_for_kbit_training\n\n# 4-bit quantization config\nbnb_config = BitsAndBytesConfig(\n    load_in_4bit=True,\n    bnb_4bit_quant_type=\"nf4\",           # NormalFloat4 (best for LLMs)\n    bnb_4bit_compute_dtype=\"bfloat16\",   # Compute in bf16\n    bnb_4bit_use_double_quant=True       # Nested quantization\n)\n\n# Load quantized model\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\u002FLlama-3.1-70B\",\n    quantization_config=bnb_config,\n    device_map=\"auto\"\n)\n\n# Prepare for training (enables gradient checkpointing)\nmodel = prepare_model_for_kbit_training(model)\n\n# LoRA config for QLoRA\nlora_config = LoraConfig(\n    r=64,                              # Higher rank for 70B\n    lora_alpha=128,\n    lora_dropout=0.1,\n    target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"],\n    bias=\"none\",\n    task_type=\"CAUSAL_LM\"\n)\n\nmodel = get_peft_model(model, lora_config)\n# 70B model now fits on single 24GB GPU!\n```\n\n## LoRA parameter selection\n\n### Rank (r) - capacity vs efficiency\n\n| Rank | Trainable Params | Memory | Quality | Use Case |\n|------|-----------------|--------|---------|----------|\n| 4 | ~3M | Minimal | Lower | Simple tasks, prototyping |\n| **8** | ~7M | Low | Good | **Recommended starting point** |\n| **16** | ~14M | Medium | Better | **General fine-tuning** |\n| 32 | ~27M | Higher | High | Complex tasks |\n| 64 | ~54M | High | Highest | Domain adaptation, 70B models |\n\n### Alpha (lora_alpha) - scaling factor\n\n```python\n# Rule of thumb: alpha = 2 * rank\nLoraConfig(r=16, lora_alpha=32)  # Standard\nLoraConfig(r=16, lora_alpha=16)  # Conservative (lower learning rate effect)\nLoraConfig(r=16, lora_alpha=64)  # Aggressive (higher learning rate effect)\n```\n\n### Target modules by architecture\n\n```python\n# Llama \u002F Mistral \u002F Qwen\ntarget_modules = [\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"]\n\n# GPT-2 \u002F GPT-Neo\ntarget_modules = [\"c_attn\", \"c_proj\", \"c_fc\"]\n\n# Falcon\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# BLOOM\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# Auto-detect all linear layers\ntarget_modules = \"all-linear\"  # PEFT 0.6.0+\n```\n\n## Loading and merging adapters\n\n### Load trained adapter\n\n```python\nfrom peft import PeftModel, AutoPeftModelForCausalLM\nfrom transformers import AutoModelForCausalLM\n\n# Option 1: Load with PeftModel\nbase_model = AutoModelForCausalLM.from_pretrained(\"meta-llama\u002FLlama-3.1-8B\")\nmodel = PeftModel.from_pretrained(base_model, \".\u002Flora-llama-adapter\")\n\n# Option 2: Load directly (recommended)\nmodel = AutoPeftModelForCausalLM.from_pretrained(\n    \".\u002Flora-llama-adapter\",\n    device_map=\"auto\"\n)\n```\n\n### Merge adapter into base model\n\n```python\n# Merge for deployment (no adapter overhead)\nmerged_model = model.merge_and_unload()\n\n# Save merged model\nmerged_model.save_pretrained(\".\u002Fllama-merged\")\ntokenizer.save_pretrained(\".\u002Fllama-merged\")\n\n# Push to Hub\nmerged_model.push_to_hub(\"username\u002Fllama-finetuned\")\n```\n\n### Multi-adapter serving\n\n```python\nfrom peft import PeftModel\n\n# Load base with first adapter\nmodel = AutoPeftModelForCausalLM.from_pretrained(\".\u002Fadapter-task1\")\n\n# Load additional adapters\nmodel.load_adapter(\".\u002Fadapter-task2\", adapter_name=\"task2\")\nmodel.load_adapter(\".\u002Fadapter-task3\", adapter_name=\"task3\")\n\n# Switch between adapters at runtime\nmodel.set_adapter(\"task1\")  # Use task1 adapter\noutput1 = model.generate(**inputs)\n\nmodel.set_adapter(\"task2\")  # Switch to task2\noutput2 = model.generate(**inputs)\n\n# Disable adapters (use base model)\nwith model.disable_adapter():\n    base_output = model.generate(**inputs)\n```\n\n## PEFT methods comparison\n\n| Method | Trainable % | Memory | Speed | Best For |\n|--------|------------|--------|-------|----------|\n| **LoRA** | 0.1-1% | Low | Fast | General fine-tuning |\n| **QLoRA** | 0.1-1% | Very Low | Medium | Memory-constrained |\n| AdaLoRA | 0.1-1% | Low | Medium | Automatic rank selection |\n| IA3 | 0.01% | Minimal | Fastest | Few-shot adaptation |\n| Prefix Tuning | 0.1% | Low | Medium | Generation control |\n| Prompt Tuning | 0.001% | Minimal | Fast | Simple task adaptation |\n| P-Tuning v2 | 0.1% | Low | Medium | NLU tasks |\n\n### IA3 (minimal parameters)\n\n```python\nfrom peft import IA3Config\n\nia3_config = IA3Config(\n    target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"down_proj\"],\n    feedforward_modules=[\"down_proj\"]\n)\nmodel = get_peft_model(model, ia3_config)\n# Trains only 0.01% of parameters!\n```\n\n### Prefix Tuning\n\n```python\nfrom peft import PrefixTuningConfig\n\nprefix_config = PrefixTuningConfig(\n    task_type=\"CAUSAL_LM\",\n    num_virtual_tokens=20,      # Prepended tokens\n    prefix_projection=True       # Use MLP projection\n)\nmodel = get_peft_model(model, prefix_config)\n```\n\n## Integration patterns\n\n### With TRL (SFTTrainer)\n\n```python\nfrom trl import SFTTrainer, SFTConfig\nfrom peft import LoraConfig\n\nlora_config = LoraConfig(r=16, lora_alpha=32, target_modules=\"all-linear\")\n\ntrainer = SFTTrainer(\n    model=model,\n    args=SFTConfig(output_dir=\".\u002Foutput\", max_seq_length=512),\n    train_dataset=dataset,\n    peft_config=lora_config,  # Pass LoRA config directly\n)\ntrainer.train()\n```\n\n### With Axolotl (YAML config)\n\n```yaml\n# axolotl config.yaml\nadapter: lora\nlora_r: 16\nlora_alpha: 32\nlora_dropout: 0.05\nlora_target_modules:\n  - q_proj\n  - v_proj\n  - k_proj\n  - o_proj\nlora_target_linear: true  # Target all linear layers\n```\n\n### With vLLM (inference)\n\n```python\nfrom vllm import LLM\nfrom vllm.lora.request import LoRARequest\n\n# Load base model with LoRA support\nllm = LLM(model=\"meta-llama\u002FLlama-3.1-8B\", enable_lora=True)\n\n# Serve with adapter\noutputs = llm.generate(\n    prompts,\n    lora_request=LoRARequest(\"adapter1\", 1, \".\u002Flora-adapter\")\n)\n```\n\n## Performance benchmarks\n\n### Memory usage (Llama 3.1 8B)\n\n| Method | GPU Memory | Trainable Params |\n|--------|-----------|------------------|\n| Full fine-tuning | 60+ GB | 8B (100%) |\n| LoRA r=16 | 18 GB | 14M (0.17%) |\n| QLoRA r=16 | 6 GB | 14M (0.17%) |\n| IA3 | 16 GB | 800K (0.01%) |\n\n### Training speed (A100 80GB)\n\n| Method | Tokens\u002Fsec | vs Full FT |\n|--------|-----------|------------|\n| Full FT | 2,500 | 1x |\n| LoRA | 3,200 | 1.3x |\n| QLoRA | 2,100 | 0.84x |\n\n### Quality (MMLU benchmark)\n\n| Model | Full FT | LoRA | QLoRA |\n|-------|---------|------|-------|\n| Llama 2-7B | 45.3 | 44.8 | 44.1 |\n| Llama 2-13B | 54.8 | 54.2 | 53.5 |\n\n---\n\n## Unsloth: 2–5× Faster LoRA\u002FQLoRA\n\nUnsloth is an optimized drop-in replacement for standard PEFT\u002FLoRA training that provides **2–5× speedup** and **30–80% VRAM reduction** while maintaining identical accuracy.\n\n### When to Use Unsloth\n\n- **Speed is critical**: Iteration cycles, large datasets, or tight deadlines\n- **VRAM-constrained hardware**: Train larger models on the same GPU (e.g., 70B on 24GB)\n- **Compatible architectures**: Llama, Mistral, Gemma, Qwen (check [docs](https:\u002F\u002Fdocs.unsloth.ai) for latest)\n\n### Installation\n\n```bash\npip install unsloth\n# Or with specific PyTorch\u002FCUDA version:\npip install \"unsloth[colab-new] @ git+https:\u002F\u002Fgithub.com\u002Funslothai\u002Funsloth.git\"\n```\n\n### Quick Start: Unsloth LoRA\n\n```python\nfrom unsloth import FastLanguageModel\n\n# Load model with automatic optimizations\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"unsloth\u002FLlama-3.2-1B\",\n    max_seq_length=2048,\n    dtype=None,          # Auto-detect (Float16\u002FBFloat16)\n    load_in_4bit=True,   # QLoRA mode\n)\n\n# Add LoRA adapter (drop-in replacement for PEFT)\nmodel = FastLanguageModel.get_peft_model(\n    model,\n    r=16,\n    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n                    \"gate_proj\", \"up_proj\", \"down_proj\"],\n    lora_alpha=16,\n    lora_dropout=0,\n    bias=\"none\",\n    use_gradient_checkpointing=\"unsloth\",  # 30% less VRAM\n    random_state=3407,\n)\n\n# Train with standard TRL or HuggingFace Trainer\nfrom trl import SFTTrainer\n\ntrainer = SFTTrainer(\n    model=model,\n    tokenizer=tokenizer,\n    train_dataset=dataset,\n    max_seq_length=2048,\n    dataset_text_field=\"text\",\n)\n\ntrainer.train()\n```\n\n### Key Optimizations\n\n| Optimization | Impact | Notes |\n|--------------|--------|-------|\n| **Manual backprop** | 2–3× speedup | Custom CUDA kernels for LoRA grads |\n| **Flash Attention 2** | 1.5–2× speedup | Enabled automatically if available |\n| **Optimized kernels** | 30% less VRAM | Fused ops, reduced intermediate allocations |\n| **GRPO support** | 2–3× speedup | `FastLanguageModel` + `GRPOTrainer` |\n\n### Unsloth + GRPO\n\n```python\nfrom unsloth import FastLanguageModel\nfrom trl import GRPOTrainer, GRPOConfig\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"unsloth\u002FQwen2.5-1.5B-Instruct\",\n    max_seq_length=1024,\n    load_in_4bit=True,\n    fast_inference=True,  # Enable for GRPO\n)\n\nmodel = FastLanguageModel.get_peft_model(\n    model, r=32, lora_alpha=32,\n    target_modules=\"all-linear\",\n    use_gradient_checkpointing=\"unsloth\",\n)\n\n# Identical GRPO workflow, 2–3× faster\ntrainer = GRPOTrainer(\n    model=model,\n    reward_funcs=[format_reward, correctness_reward],\n    args=GRPOConfig(output_dir=\".\u002Fgrpo-unsloth\", num_generations=8),\n    train_dataset=dataset,\n)\ntrainer.train()\n```\n\n### Trade-offs\n\n- **Architecture coverage**: Only Llama, Mistral, Gemma, Qwen, and a few others (not universal)\n- **Newer method**: Less battle-tested than vanilla PEFT in some edge cases\n- **Installation**: May require specific CUDA\u002FPyTorch versions\n\n### Resources\n\n- **Docs**: https:\u002F\u002Fdocs.unsloth.ai\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Funslothai\u002Funsloth\n- **Colab**: https:\u002F\u002Fcolab.research.google.com\u002Fdrive\u002F1Dyauq4kTZoLewQ1cApceUQVNcnnNTzg_\n\n## Common issues\n\n### CUDA OOM during training\n\n```python\n# Solution 1: Enable gradient checkpointing\nmodel.gradient_checkpointing_enable()\n\n# Solution 2: Reduce batch size + increase accumulation\nTrainingArguments(\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=16\n)\n\n# Solution 3: Use QLoRA\nfrom transformers import BitsAndBytesConfig\nbnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\")\n```\n\n### Adapter not applying\n\n```python\n# Verify adapter is active\nprint(model.active_adapters)  # Should show adapter name\n\n# Check trainable parameters\nmodel.print_trainable_parameters()\n\n# Ensure model in training mode\nmodel.train()\n```\n\n### Quality degradation\n\n```python\n# Increase rank\nLoraConfig(r=32, lora_alpha=64)\n\n# Target more modules\ntarget_modules = \"all-linear\"\n\n# Use more training data and epochs\nTrainingArguments(num_train_epochs=5)\n\n# Lower learning rate\nTrainingArguments(learning_rate=1e-4)\n```\n\n## Best practices\n\n1. **Start with r=8-16**, increase if quality insufficient\n2. **Use alpha = 2 * rank** as starting point\n3. **Target attention + MLP layers** for best quality\u002Fefficiency\n4. **Enable gradient checkpointing** for memory savings\n5. **Save adapters frequently** (small files, easy rollback)\n6. **Evaluate on held-out data** before merging\n7. **Use QLoRA for 70B+ models** on consumer hardware\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - DoRA, LoftQ, rank stabilization, custom modules\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common errors, debugging, optimization\n\n## Resources\n\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fpeft\n- **Docs**: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fpeft\n- **LoRA Paper**: arXiv:2106.09685\n- **QLoRA Paper**: arXiv:2305.14314\n- **Models**: https:\u002F\u002Fhuggingface.co\u002Fmodels?library=peft\n","# PEFT (Parameter-Efficient Fine-Tuning)\n\nFine-tune LLMs by training \u003C1% of parameters using LoRA, QLoRA, and 25+ adapter methods.\n\n## When to use PEFT\n\n**Use PEFT\u002FLoRA when:**\n- Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100)\n- Need to train \u003C1% parameters (6MB adapters vs 14GB full model)\n- Want fast iteration with multiple task-specific adapters\n- Deploying multiple fine-tuned variants from one base model\n\n**Use QLoRA (PEFT + quantization) when:**\n- Fine-tuning 70B models on single 24GB GPU\n- Memory is the primary constraint\n- Can accept ~5% quality trade-off vs full fine-tuning\n\n**Use full fine-tuning instead when:**\n- Training small models (\u003C1B parameters)\n- Need maximum quality and have compute budget\n- Significant domain shift requires updating all weights\n\n## Quick start\n\n### Installation\n\n```bash\n# Basic installation\npip install peft\n\n# With quantization support (recommended)\npip install peft bitsandbytes\n\n# Full stack\npip install peft transformers accelerate bitsandbytes datasets\n```\n\n### LoRA fine-tuning (standard)\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer\nfrom peft import get_peft_model, LoraConfig, TaskType\nfrom datasets import load_dataset\n\n# Load base model\nmodel_name = \"meta-llama\u002FLlama-3.1-8B\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=\"auto\", device_map=\"auto\")\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# LoRA configuration\nlora_config = LoraConfig(\n    task_type=TaskType.CAUSAL_LM,\n    r=16,                          # Rank (8-64, higher = more capacity)\n    lora_alpha=32,                 # Scaling factor (typically 2*r)\n    lora_dropout=0.05,             # Dropout for regularization\n    target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\"],  # Attention layers\n    bias=\"none\"                    # Don't train biases\n)\n\n# Apply LoRA\nmodel = get_peft_model(model, lora_config)\nmodel.print_trainable_parameters()\n# Output: trainable params: 13,631,488 || all params: 8,043,307,008 || trainable%: 0.17%\n\n# Prepare dataset\ndataset = load_dataset(\"databricks\u002Fdatabricks-dolly-15k\", split=\"train\")\n\ndef tokenize(example):\n    text = f\"### Instruction:\\n{example['instruction']}\\n\\n### Response:\\n{example['response']}\"\n    return tokenizer(text, truncation=True, max_length=512, padding=\"max_length\")\n\ntokenized = dataset.map(tokenize, remove_columns=dataset.column_names)\n\n# Training\ntraining_args = TrainingArguments(\n    output_dir=\".\u002Flora-llama\",\n    num_train_epochs=3,\n    per_device_train_batch_size=4,\n    gradient_accumulation_steps=4,\n    learning_rate=2e-4,\n    fp16=True,\n    logging_steps=10,\n    save_strategy=\"epoch\"\n)\n\ntrainer = Trainer(\n    model=model,\n    args=training_args,\n    train_dataset=tokenized,\n    data_collator=lambda data: {\"input_ids\": torch.stack([f[\"input_ids\"] for f in data]),\n                                 \"attention_mask\": torch.stack([f[\"attention_mask\"] for f in data]),\n                                 \"labels\": torch.stack([f[\"input_ids\"] for f in data])}\n)\n\ntrainer.train()\n\n# Save adapter only (6MB vs 16GB)\nmodel.save_pretrained(\".\u002Flora-llama-adapter\")\n```\n\n### QLoRA fine-tuning (memory-efficient)\n\n```python\nfrom transformers import AutoModelForCausalLM, BitsAndBytesConfig\nfrom peft import get_peft_model, LoraConfig, prepare_model_for_kbit_training\n\n# 4-bit quantization config\nbnb_config = BitsAndBytesConfig(\n    load_in_4bit=True,\n    bnb_4bit_quant_type=\"nf4\",           # NormalFloat4 (best for LLMs)\n    bnb_4bit_compute_dtype=\"bfloat16\",   # Compute in bf16\n    bnb_4bit_use_double_quant=True       # Nested quantization\n)\n\n# Load quantized model\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\u002FLlama-3.1-70B\",\n    quantization_config=bnb_config,\n    device_map=\"auto\"\n)\n\n# Prepare for training (enables gradient checkpointing)\nmodel = prepare_model_for_kbit_training(model)\n\n# LoRA config for QLoRA\nlora_config = LoraConfig(\n    r=64,                              # Higher rank for 70B\n    lora_alpha=128,\n    lora_dropout=0.1,\n    target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"],\n    bias=\"none\",\n    task_type=\"CAUSAL_LM\"\n)\n\nmodel = get_peft_model(model, lora_config)\n# 70B model now fits on single 24GB GPU!\n```\n\n## LoRA parameter selection\n\n### Rank (r) - capacity vs efficiency\n\n| Rank | Trainable Params | Memory | Quality | Use Case |\n|------|-----------------|--------|---------|----------|\n| 4 | ~3M | Minimal | Lower | Simple tasks, prototyping |\n| **8** | ~7M | Low | Good | **Recommended starting point** |\n| **16** | ~14M | Medium | Better | **General fine-tuning** |\n| 32 | ~27M | Higher | High | Complex tasks |\n| 64 | ~54M | High | Highest | Domain adaptation, 70B models |\n\n### Alpha (lora_alpha) - scaling factor\n\n```python\n# Rule of thumb: alpha = 2 * rank\nLoraConfig(r=16, lora_alpha=32)  # Standard\nLoraConfig(r=16, lora_alpha=16)  # Conservative (lower learning rate effect)\nLoraConfig(r=16, lora_alpha=64)  # Aggressive (higher learning rate effect)\n```\n\n### Target modules by architecture\n\n```python\n# Llama \u002F Mistral \u002F Qwen\ntarget_modules = [\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"]\n\n# GPT-2 \u002F GPT-Neo\ntarget_modules = [\"c_attn\", \"c_proj\", \"c_fc\"]\n\n# Falcon\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# BLOOM\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# Auto-detect all linear layers\ntarget_modules = \"all-linear\"  # PEFT 0.6.0+\n```\n\n## Loading and merging adapters\n\n### Load trained adapter\n\n```python\nfrom peft import PeftModel, AutoPeftModelForCausalLM\nfrom transformers import AutoModelForCausalLM\n\n# Option 1: Load with PeftModel\nbase_model = AutoModelForCausalLM.from_pretrained(\"meta-llama\u002FLlama-3.1-8B\")\nmodel = PeftModel.from_pretrained(base_model, \".\u002Flora-llama-adapter\")\n\n# Option 2: Load directly (recommended)\nmodel = AutoPeftModelForCausalLM.from_pretrained(\n    \".\u002Flora-llama-adapter\",\n    device_map=\"auto\"\n)\n```\n\n### Merge adapter into base model\n\n```python\n# Merge for deployment (no adapter overhead)\nmerged_model = model.merge_and_unload()\n\n# Save merged model\nmerged_model.save_pretrained(\".\u002Fllama-merged\")\ntokenizer.save_pretrained(\".\u002Fllama-merged\")\n\n# Push to Hub\nmerged_model.push_to_hub(\"username\u002Fllama-finetuned\")\n```\n\n### Multi-adapter serving\n\n```python\nfrom peft import PeftModel\n\n# Load base with first adapter\nmodel = AutoPeftModelForCausalLM.from_pretrained(\".\u002Fadapter-task1\")\n\n# Load additional adapters\nmodel.load_adapter(\".\u002Fadapter-task2\", adapter_name=\"task2\")\nmodel.load_adapter(\".\u002Fadapter-task3\", adapter_name=\"task3\")\n\n# Switch between adapters at runtime\nmodel.set_adapter(\"task1\")  # Use task1 adapter\noutput1 = model.generate(**inputs)\n\nmodel.set_adapter(\"task2\")  # Switch to task2\noutput2 = model.generate(**inputs)\n\n# Disable adapters (use base model)\nwith model.disable_adapter():\n    base_output = model.generate(**inputs)\n```\n\n## PEFT methods comparison\n\n| Method | Trainable % | Memory | Speed | Best For |\n|--------|------------|--------|-------|----------|\n| **LoRA** | 0.1-1% | Low | Fast | General fine-tuning |\n| **QLoRA** | 0.1-1% | Very Low | Medium | Memory-constrained |\n| AdaLoRA | 0.1-1% | Low | Medium | Automatic rank selection |\n| IA3 | 0.01% | Minimal | Fastest | Few-shot adaptation |\n| Prefix Tuning | 0.1% | Low | Medium | Generation control |\n| Prompt Tuning | 0.001% | Minimal | Fast | Simple task adaptation |\n| P-Tuning v2 | 0.1% | Low | Medium | NLU tasks |\n\n### IA3 (minimal parameters)\n\n```python\nfrom peft import IA3Config\n\nia3_config = IA3Config(\n    target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"down_proj\"],\n    feedforward_modules=[\"down_proj\"]\n)\nmodel = get_peft_model(model, ia3_config)\n# Trains only 0.01% of parameters!\n```\n\n### Prefix Tuning\n\n```python\nfrom peft import PrefixTuningConfig\n\nprefix_config = PrefixTuningConfig(\n    task_type=\"CAUSAL_LM\",\n    num_virtual_tokens=20,      # Prepended tokens\n    prefix_projection=True       # Use MLP projection\n)\nmodel = get_peft_model(model, prefix_config)\n```\n\n## Integration patterns\n\n### With TRL (SFTTrainer)\n\n```python\nfrom trl import SFTTrainer, SFTConfig\nfrom peft import LoraConfig\n\nlora_config = LoraConfig(r=16, lora_alpha=32, target_modules=\"all-linear\")\n\ntrainer = SFTTrainer(\n    model=model,\n    args=SFTConfig(output_dir=\".\u002Foutput\", max_seq_length=512),\n    train_dataset=dataset,\n    peft_config=lora_config,  # Pass LoRA config directly\n)\ntrainer.train()\n```\n\n### With Axolotl (YAML config)\n\n```yaml\n# axolotl config.yaml\nadapter: lora\nlora_r: 16\nlora_alpha: 32\nlora_dropout: 0.05\nlora_target_modules:\n  - q_proj\n  - v_proj\n  - k_proj\n  - o_proj\nlora_target_linear: true  # Target all linear layers\n```\n\n### With vLLM (inference)\n\n```python\nfrom vllm import LLM\nfrom vllm.lora.request import LoRARequest\n\n# Load base model with LoRA support\nllm = LLM(model=\"meta-llama\u002FLlama-3.1-8B\", enable_lora=True)\n\n# Serve with adapter\noutputs = llm.generate(\n    prompts,\n    lora_request=LoRARequest(\"adapter1\", 1, \".\u002Flora-adapter\")\n)\n```\n\n## Performance benchmarks\n\n### Memory usage (Llama 3.1 8B)\n\n| Method | GPU Memory | Trainable Params |\n|--------|-----------|------------------|\n| Full fine-tuning | 60+ GB | 8B (100%) |\n| LoRA r=16 | 18 GB | 14M (0.17%) |\n| QLoRA r=16 | 6 GB | 14M (0.17%) |\n| IA3 | 16 GB | 800K (0.01%) |\n\n### Training speed (A100 80GB)\n\n| Method | Tokens\u002Fsec | vs Full FT |\n|--------|-----------|------------|\n| Full FT | 2,500 | 1x |\n| LoRA | 3,200 | 1.3x |\n| QLoRA | 2,100 | 0.84x |\n\n### Quality (MMLU benchmark)\n\n| Model | Full FT | LoRA | QLoRA |\n|-------|---------|------|-------|\n| Llama 2-7B | 45.3 | 44.8 | 44.1 |\n| Llama 2-13B | 54.8 | 54.2 | 53.5 |\n\n---\n\n## Unsloth: 2–5× Faster LoRA\u002FQLoRA\n\nUnsloth is an optimized drop-in replacement for standard PEFT\u002FLoRA training that provides **2–5× speedup** and **30–80% VRAM reduction** while maintaining identical accuracy.\n\n### When to Use Unsloth\n\n- **Speed is critical**: Iteration cycles, large datasets, or tight deadlines\n- **VRAM-constrained hardware**: Train larger models on the same GPU (e.g., 70B on 24GB)\n- **Compatible architectures**: Llama, Mistral, Gemma, Qwen (check [docs](https:\u002F\u002Fdocs.unsloth.ai) for latest)\n\n### Installation\n\n```bash\npip install unsloth\n# Or with specific PyTorch\u002FCUDA version:\npip install \"unsloth[colab-new] @ git+https:\u002F\u002Fgithub.com\u002Funslothai\u002Funsloth.git\"\n```\n\n### Quick Start: Unsloth LoRA\n\n```python\nfrom unsloth import FastLanguageModel\n\n# Load model with automatic optimizations\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"unsloth\u002FLlama-3.2-1B\",\n    max_seq_length=2048,\n    dtype=None,          # Auto-detect (Float16\u002FBFloat16)\n    load_in_4bit=True,   # QLoRA mode\n)\n\n# Add LoRA adapter (drop-in replacement for PEFT)\nmodel = FastLanguageModel.get_peft_model(\n    model,\n    r=16,\n    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n                    \"gate_proj\", \"up_proj\", \"down_proj\"],\n    lora_alpha=16,\n    lora_dropout=0,\n    bias=\"none\",\n    use_gradient_checkpointing=\"unsloth\",  # 30% less VRAM\n    random_state=3407,\n)\n\n# Train with standard TRL or HuggingFace Trainer\nfrom trl import SFTTrainer\n\ntrainer = SFTTrainer(\n    model=model,\n    tokenizer=tokenizer,\n    train_dataset=dataset,\n    max_seq_length=2048,\n    dataset_text_field=\"text\",\n)\n\ntrainer.train()\n```\n\n### Key Optimizations\n\n| Optimization | Impact | Notes |\n|--------------|--------|-------|\n| **Manual backprop** | 2–3× speedup | Custom CUDA kernels for LoRA grads |\n| **Flash Attention 2** | 1.5–2× speedup | Enabled automatically if available |\n| **Optimized kernels** | 30% less VRAM | Fused ops, reduced intermediate allocations |\n| **GRPO support** | 2–3× speedup | `FastLanguageModel` + `GRPOTrainer` |\n\n### Unsloth + GRPO\n\n```python\nfrom unsloth import FastLanguageModel\nfrom trl import GRPOTrainer, GRPOConfig\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"unsloth\u002FQwen2.5-1.5B-Instruct\",\n    max_seq_length=1024,\n    load_in_4bit=True,\n    fast_inference=True,  # Enable for GRPO\n)\n\nmodel = FastLanguageModel.get_peft_model(\n    model, r=32, lora_alpha=32,\n    target_modules=\"all-linear\",\n    use_gradient_checkpointing=\"unsloth\",\n)\n\n# Identical GRPO workflow, 2–3× faster\ntrainer = GRPOTrainer(\n    model=model,\n    reward_funcs=[format_reward, correctness_reward],\n    args=GRPOConfig(output_dir=\".\u002Fgrpo-unsloth\", num_generations=8),\n    train_dataset=dataset,\n)\ntrainer.train()\n```\n\n### Trade-offs\n\n- **Architecture coverage**: Only Llama, Mistral, Gemma, Qwen, and a few others (not universal)\n- **Newer method**: Less battle-tested than vanilla PEFT in some edge cases\n- **Installation**: May require specific CUDA\u002FPyTorch versions\n\n### Resources\n\n- **Docs**: https:\u002F\u002Fdocs.unsloth.ai\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Funslothai\u002Funsloth\n- **Colab**: https:\u002F\u002Fcolab.research.google.com\u002Fdrive\u002F1Dyauq4kTZoLewQ1cApceUQVNcnnNTzg_\n\n## Common issues\n\n### CUDA OOM during training\n\n```python\n# Solution 1: Enable gradient checkpointing\nmodel.gradient_checkpointing_enable()\n\n# Solution 2: Reduce batch size + increase accumulation\nTrainingArguments(\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=16\n)\n\n# Solution 3: Use QLoRA\nfrom transformers import BitsAndBytesConfig\nbnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\")\n```\n\n### Adapter not applying\n\n```python\n# Verify adapter is active\nprint(model.active_adapters)  # Should show adapter name\n\n# Check trainable parameters\nmodel.print_trainable_parameters()\n\n# Ensure model in training mode\nmodel.train()\n```\n\n### Quality degradation\n\n```python\n# Increase rank\nLoraConfig(r=32, lora_alpha=64)\n\n# Target more modules\ntarget_modules = \"all-linear\"\n\n# Use more training data and epochs\nTrainingArguments(num_train_epochs=5)\n\n# Lower learning rate\nTrainingArguments(learning_rate=1e-4)\n```\n\n## Best practices\n\n1. **Start with r=8-16**, increase if quality insufficient\n2. **Use alpha = 2 * rank** as starting point\n3. **Target attention + MLP layers** for best quality\u002Fefficiency\n4. **Enable gradient checkpointing** for memory savings\n5. **Save adapters frequently** (small files, easy rollback)\n6. **Evaluate on held-out data** before merging\n7. **Use QLoRA for 70B+ models** on consumer hardware\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - DoRA, LoftQ, rank stabilization, custom modules\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common errors, debugging, optimization\n\n## Resources\n\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fpeft\n- **Docs**: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fpeft\n- **LoRA Paper**: arXiv:2106.09685\n- **QLoRA Paper**: arXiv:2305.14314\n- **Models**: https:\u002F\u002Fhuggingface.co\u002Fmodels?library=peft\n",{"name":5,"author":43,"license":15,"version":44,"metadata":45,"description":7,"dependencies":54},"Orchestra Research","1.0.0",{"hermes":46},{"tags":47},[31,48,30,22,49,50,51,52,53],"PEFT","Parameter-Efficient","Adapters","Low-Rank","Memory Optimization","Multi-Adapter",[55,56,57,58],"peft>=0.13.0","transformers>=4.45.0","torch>=2.0.0","bitsandbytes>=0.43.0","main","64e2fc9afa7bb3da1138eb73399144d0e7e73e9d","_hermes\u002Fdefault\u002Fprofiles\u002Fwriting\u002Fskills\u002Fmlops\u002Ftraining\u002Fpeft",4,"https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs","https:\u002F\u002Fgithub.com\u002Fadnichols","2026-11-06 00:28:38",{"source":8,"skillName":5,"commands":67,"prompt":77,"zipAvailable":78},[68,71,74],{"tool":69,"command":70},"npx","npx skills add adnichols\u002Fai-configs --skill peft-fine-tuning",{"tool":72,"command":73},"bunx","bunx skills add adnichols\u002Fai-configs --skill peft-fine-tuning",{"tool":75,"command":76},"pnpm","pnpm dlx skills add adnichols\u002Fai-configs --skill peft-fine-tuning","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fprofiles\u002Fwriting\u002Fskills\u002Fmlops\u002Ftraining\u002Fpeft\u002FSKILL.md",true,[80,86],{"path":81,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":84,"sourceHtmlUrl":85},"references\u002Fadvanced-usage.md","REFERENCE","参考文档",12541,"https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fprofiles\u002Fwriting\u002Fskills\u002Fmlops\u002Ftraining\u002Fpeft\u002Freferences\u002Fadvanced-usage.md",{"path":87,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":88,"sourceHtmlUrl":89},"references\u002Ftroubleshooting.md",10344,"https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fprofiles\u002Fwriting\u002Fskills\u002Fmlops\u002Ftraining\u002Fpeft\u002Freferences\u002Ftroubleshooting.md",[91,111,145,170,181,197],{"skillNo":92,"name":93,"nameZh":6,"description":94,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":38},"01M6CB7RN791P5RRRCJWWC7DAR","blocked-page-recovery","Use when a fetch fails: 403\u002F429, paywall, WAF, bot wall.","https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fskills\u002Fweb\u002Fblocked-page-recovery\u002FSKILL.md",[],[98,99,100,101,102,25,103,104,105,106,107,108,36,109,110],"recovery","page","blocked page recovery","bot","fallback","fetch","hermes agent","hermes","Citations","waf","windows","linux","macos",{"skillNo":112,"name":113,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":115,"categories":116,"keywords":130,"publishedAt":38},"01M6CB7NQY2VVYCQJ1QBXACQF4","writing-plans","Use when creating or updating executable software plans. Keep plans as local files unless the user explicitly asks to register them in Doct.","https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fskills\u002Fsoftware-development\u002Fwriting-plans\u002FSKILL.md",[117,122,125],{"categoryNo":118,"categoryName":119,"taxonomyType":120,"taxonomyTypeDesc":121},"01JBSEEDCAT000000000000018","写作与文案","SCENARIO","使用场景",{"categoryNo":123,"categoryName":124,"taxonomyType":120,"taxonomyTypeDesc":121},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":126,"categoryName":127,"taxonomyType":128,"taxonomyTypeDesc":129},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[131,132,133,134,135,136,137,138,139,140,141,142,25,143,104,105,144],"writing","writing plans","documentation","test","workflow","design","user","development","planning","files","implementation","html","ops","plan",{"skillNo":146,"name":147,"nameZh":6,"description":148,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":149,"categories":150,"keywords":160,"publishedAt":38},"01M6CB7JME246KZQHP73Z2STDF","test-driven-development","TDD: enforce RED-GREEN-REFACTOR, tests before code.","https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fskills\u002Fsoftware-development\u002Ftest-driven-development\u002FSKILL.md",[151,154,157],{"categoryNo":152,"categoryName":153,"taxonomyType":120,"taxonomyTypeDesc":121},"01JBSEEDCAT000000000000007","代码审查",{"categoryNo":155,"categoryName":156,"taxonomyType":120,"taxonomyTypeDesc":121},"01JBSEEDCAT000000000000008","测试与质量",{"categoryNo":158,"categoryName":159,"taxonomyType":128,"taxonomyTypeDesc":129},"01JBSEEDCAT000000000000037","律师与法务",[134,138,161,162,163,104,105,164,144,165,166,167,110,168,109,169,108],"test driven development","red-green-refactor","superpowers","quality","refactor","code","debugging","tdd","testing",{"skillNo":171,"name":172,"nameZh":6,"description":173,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":174,"categories":175,"keywords":177,"publishedAt":38},"01M6CB7FQ407KQTD4Z9VAWKY54","systematic-debugging","4-phase root cause debugging: understand bugs before fixing.","https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fskills\u002Fsoftware-development\u002Fsystematic-debugging\u002FSKILL.md",[176],{"categoryNo":155,"categoryName":156,"taxonomyType":120,"taxonomyTypeDesc":121},[167,178,138,163,104,105,179,144,180,108,134,109,110],"systematic debugging","troubleshooting","investigation",{"skillNo":182,"name":183,"nameZh":6,"description":184,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":185,"categories":186,"keywords":190,"publishedAt":38},"01M6CB7D57MK88HE5YPZHKH68Q","spike","Throwaway experiments to validate an idea before build.","https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fskills\u002Fsoftware-development\u002Fspike\u002FSKILL.md",[187],{"categoryNo":188,"categoryName":189,"taxonomyType":120,"taxonomyTypeDesc":121},"01JBSEEDCAT000000000000021","资料检索与研究",[183,108,191,192,36,193,109,110,194,195,196,104,105,144,139],"validate","experiments","prototype","feasibility","sketch","experiment",{"skillNo":198,"name":199,"nameZh":6,"description":200,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":201,"categories":202,"keywords":207,"publishedAt":38},"01M6CB7APAQ37YE5YE1MAAFT42","simplify-code","Parallel 3-agent cleanup of recent code changes.","https:\u002F\u002Fgithub.com\u002Fadnichols\u002Fai-configs\u002Fblob\u002Fmain\u002F_hermes\u002Fdefault\u002Fskills\u002Fsoftware-development\u002Fsimplify-code\u002FSKILL.md",[203,204],{"categoryNo":123,"categoryName":124,"taxonomyType":120,"taxonomyTypeDesc":121},{"categoryNo":205,"categoryName":206,"taxonomyType":128,"taxonomyTypeDesc":129},"01JBSEEDCAT000000000000030","开发工程师",[208,209,166,210,104,105,211,144,165,212,213,214,138,108,134,109,215,110],"simplify code","simplify","subagent","parallel","code-review","delegation","cleanup","review"]