[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M74Z14S76E7PRMAV2AP3M5T6":3,"skill-same:01M74Z14S76E7PRMAV2AP3M5T6":64},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":33,"repositoryHtmlUrl":34,"repositoryHomepageUrl":35,"ownerHtmlUrl":36,"ownerBlogUrl":37,"ownerDescription":38,"updatedAt":39,"install":40,"resources":53},"01M74Z14S76E7PRMAV2AP3M5T6","model-pruning",null,"Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.","KalarisLabs\u002Fresearch-agent-skills","KalarisLabs","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F300878568?v=4","01M74Y7MT23GXXC56VM6TVSXZY","Kalarislabs","01M74Y7MTQM9REXZX4SNXWX95C",132,"MIT","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fmodel-pruning\u002FSKILL.md",[],[],"2026-11-12 13:55:43",false,"---\nname: model-pruning\ndescription: Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: multimodal-and-emerging\n  maintainer: Kalaris Labs\n  tags: Emerging Techniques, Model Pruning, Wanda, SparseGPT, Sparsity, Model Compression, N:M Sparsity, One-Shot Pruning, Structured Pruning, Unstructured Pruning, Fast Inference\n  dependencies: transformers, torch\n---\n\n# Model Pruning: Compressing LLMs\n\n## When to Use This Skill\n\nUse Model Pruning when you need to:\n- **Reduce model size** by 40-60% with \u003C1% accuracy loss\n- **Accelerate inference** using hardware-friendly sparsity (2-4× speedup)\n- **Deploy on constrained hardware** (mobile, edge devices)\n- **Compress without retraining** using one-shot methods\n- **Enable efficient serving** with reduced memory footprint\n\n**Key Techniques**: Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity\n\n**Papers**: Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774)\n\n## Installation\n\n```bash\n# Wanda implementation\ngit clone https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\ncd wanda\npip install -r requirements.txt\n\n# Optional: SparseGPT\ngit clone https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\ncd sparsegpt\npip install -e .\n\n# Dependencies\npip install torch transformers accelerate\n```\n\n## Quick Start\n\n### Wanda Pruning (One-Shot, No Retraining)\n\n**Source**: ICLR 2024 (arXiv 2306.11695)\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\u002FLlama-2-7b-hf\",\n    torch_dtype=torch.float16,\n    device_map=\"cuda\"\n)\ntokenizer = AutoTokenizer.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Calibration data (small dataset for activation statistics)\ncalib_data = [\n    \"The quick brown fox jumps over the lazy dog.\",\n    \"Machine learning is transforming the world.\",\n    \"Artificial intelligence powers modern applications.\",\n]\n\n# Wanda pruning function\ndef wanda_prune(model, calib_data, sparsity=0.5):\n    \"\"\"\n    Wanda: Prune by weight magnitude × input activation.\n\n    Args:\n        sparsity: Fraction of weights to prune (0.5 = 50%)\n    \"\"\"\n    # 1. Collect activation statistics\n    activations = {}\n\n    def hook_fn(name):\n        def hook(module, input, output):\n            # Store input activation norms\n            activations[name] = input[0].detach().abs().mean(dim=0)\n        return hook\n\n    # Register hooks for all linear layers\n    hooks = []\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            hooks.append(module.register_forward_hook(hook_fn(name)))\n\n    # Run calibration data\n    model.eval()\n    with torch.no_grad():\n        for text in calib_data:\n            inputs = tokenizer(text, return_tensors=\"pt\").to(model.device)\n            model(**inputs)\n\n    # Remove hooks\n    for hook in hooks:\n        hook.remove()\n\n    # 2. Prune weights based on |weight| × activation\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear) and name in activations:\n            W = module.weight.data\n            act = activations[name]\n\n            # Compute importance: |weight| × activation\n            importance = W.abs() * act.unsqueeze(0)\n\n            # Flatten and find threshold\n            threshold = torch.quantile(importance.flatten(), sparsity)\n\n            # Create mask\n            mask = importance >= threshold\n\n            # Apply mask (prune)\n            W *= mask.float()\n\n    return model\n\n# Apply Wanda pruning (50% sparsity, one-shot, no retraining)\npruned_model = wanda_prune(model, calib_data, sparsity=0.5)\n\n# Save\npruned_model.save_pretrained(\".\u002Fllama-2-7b-wanda-50\")\n```\n\n### SparseGPT (Second-Order Pruning)\n\n**Source**: arXiv 2301.00774\n\n```python\nfrom sparsegpt import SparseGPT\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Initialize SparseGPT\npruner = SparseGPT(model)\n\n# Calibration data\ncalib_data = load_calibration_data()  # ~128 samples\n\n# Prune (one-shot, layer-wise reconstruction)\npruned_model = pruner.prune(\n    calib_data=calib_data,\n    sparsity=0.5,           # 50% sparsity\n    prunen=0,               # Unstructured (0) or N:M structured\n    prunem=0,\n    percdamp=0.01,          # Damping for Hessian inverse\n)\n\n# Results: Near-lossless pruning at 50% sparsity\n```\n\n### N:M Structured Pruning (Hardware Accelerator)\n\n```python\ndef nm_prune(weight, n=2, m=4):\n    \"\"\"\n    N:M pruning: Keep N weights per M consecutive weights.\n    Example: 2:4 = keep 2 out of every 4 weights.\n\n    Compatible with NVIDIA sparse tensor cores (2:4, 4:8).\n    \"\"\"\n    # Reshape weight into groups of M\n    shape = weight.shape\n    weight_flat = weight.flatten()\n\n    # Pad to multiple of M\n    pad_size = (m - weight_flat.numel() % m) % m\n    weight_padded = F.pad(weight_flat, (0, pad_size))\n\n    # Reshape into (num_groups, m)\n    weight_grouped = weight_padded.reshape(-1, m)\n\n    # Find top-N in each group\n    _, indices = torch.topk(weight_grouped.abs(), n, dim=-1)\n\n    # Create mask\n    mask = torch.zeros_like(weight_grouped)\n    mask.scatter_(1, indices, 1.0)\n\n    # Apply mask\n    weight_pruned = weight_grouped * mask\n\n    # Reshape back\n    weight_pruned = weight_pruned.flatten()[:weight_flat.numel()]\n    return weight_pruned.reshape(shape)\n\n# Apply 2:4 sparsity (NVIDIA hardware)\nfor name, module in model.named_modules():\n    if isinstance(module, torch.nn.Linear):\n        module.weight.data = nm_prune(module.weight.data, n=2, m=4)\n\n# 50% sparsity, 2× speedup on A100 with sparse tensor cores\n```\n\n## Core Concepts\n\n### 1. Pruning Criteria\n\n**Magnitude Pruning** (baseline):\n```python\n# Prune weights with smallest absolute values\nimportance = weight.abs()\nthreshold = torch.quantile(importance, sparsity)\nmask = importance >= threshold\n```\n\n**Wanda** (weights × activations):\n```python\n# Importance = |weight| × input_activation\nimportance = weight.abs() * activation\n# Better than magnitude alone (considers usage)\n```\n\n**SparseGPT** (second-order):\n```python\n# Uses Hessian (second derivative) for importance\n# More accurate but computationally expensive\nimportance = weight^2 \u002F diag(Hessian)\n```\n\n### 2. Structured vs Unstructured\n\n**Unstructured** (fine-grained):\n- Prune individual weights\n- Higher quality (better accuracy)\n- No hardware speedup (irregular sparsity)\n\n**Structured** (coarse-grained):\n- Prune entire neurons, heads, or layers\n- Lower quality (more accuracy loss)\n- Hardware speedup (regular sparsity)\n\n**Semi-structured (N:M)**:\n- Best of both worlds\n- 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs\n- Minimal accuracy loss\n\n### 3. Sparsity Patterns\n\n```python\n# Unstructured (random)\n# [1, 0, 1, 0, 1, 1, 0, 0]\n# Pros: Flexible, high quality\n# Cons: No speedup\n\n# Structured (block)\n# [1, 1, 0, 0, 1, 1, 0, 0]\n# Pros: Hardware friendly\n# Cons: More accuracy loss\n\n# N:M (semi-structured)\n# [1, 0, 1, 0] [1, 1, 0, 0]  (2:4 pattern)\n# Pros: Hardware speedup + good quality\n# Cons: Requires specific hardware (NVIDIA)\n```\n\n## Pruning Strategies\n\n### Strategy 1: Gradual Magnitude Pruning\n\n```python\ndef gradual_prune(model, initial_sparsity=0.0, final_sparsity=0.5, num_steps=100):\n    \"\"\"Gradually increase sparsity during training.\"\"\"\n    for step in range(num_steps):\n        # Current sparsity\n        current_sparsity = initial_sparsity + (final_sparsity - initial_sparsity) * (step \u002F num_steps)\n\n        # Prune at current sparsity\n        for module in model.modules():\n            if isinstance(module, torch.nn.Linear):\n                weight = module.weight.data\n                threshold = torch.quantile(weight.abs().flatten(), current_sparsity)\n                mask = weight.abs() >= threshold\n                weight *= mask.float()\n\n        # Train one step\n        train_step(model)\n\n    return model\n```\n\n### Strategy 2: Layer-wise Pruning\n\n```python\ndef layer_wise_prune(model, sparsity_per_layer):\n    \"\"\"Different sparsity for different layers.\"\"\"\n    # Early layers: Less pruning (more important)\n    # Late layers: More pruning (less critical)\n\n    sparsity_schedule = {\n        \"layer.0\": 0.3,   # 30% sparsity\n        \"layer.1\": 0.4,\n        \"layer.2\": 0.5,\n        \"layer.3\": 0.6,   # 60% sparsity\n    }\n\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            # Find layer index\n            for layer_name, sparsity in sparsity_schedule.items():\n                if layer_name in name:\n                    # Prune at layer-specific sparsity\n                    prune_layer(module, sparsity)\n                    break\n\n    return model\n```\n\n### Strategy 3: Iterative Pruning + Fine-tuning\n\n```python\ndef iterative_prune_finetune(model, target_sparsity=0.5, iterations=5):\n    \"\"\"Prune gradually with fine-tuning between iterations.\"\"\"\n    current_sparsity = 0.0\n    sparsity_increment = target_sparsity \u002F iterations\n\n    for i in range(iterations):\n        # Increase sparsity\n        current_sparsity += sparsity_increment\n\n        # Prune\n        prune_model(model, sparsity=current_sparsity)\n\n        # Fine-tune (recover accuracy)\n        fine_tune(model, epochs=2, lr=1e-5)\n\n    return model\n\n# Results: Better accuracy than one-shot at high sparsity\n```\n\n## Production Deployment\n\nDetails, code examples and parameter tables: [references\u002Fproduction-deployment.md](references\u002Fproduction-deployment.md). Read it when this step applies.\n\n## Best Practices\n\n### 1. Sparsity Selection\n\n```python\n# Conservative (safe)\nsparsity = 0.3  # 30%, \u003C0.5% loss\n\n# Balanced (recommended)\nsparsity = 0.5  # 50%, ~1% loss\n\n# Aggressive (risky)\nsparsity = 0.7  # 70%, 2-5% loss\n\n# Extreme (model-dependent)\nsparsity = 0.9  # 90%, significant degradation\n```\n\n### 2. Method Selection\n\n```python\n# One-shot, no retraining → Wanda or SparseGPT\nif no_retraining_budget:\n    use_method = \"wanda\"  # Faster\n\n# Best quality → SparseGPT\nif need_best_quality:\n    use_method = \"sparsegpt\"  # More accurate\n\n# Hardware speedup → N:M structured\nif need_speedup:\n    use_method = \"nm_prune\"  # 2:4 or 4:8\n```\n\n### 3. Avoid Common Pitfalls\n\n```python\n# ❌ Bad: Pruning without calibration data\nprune_random(model)  # No activation statistics\n\n# ✅ Good: Use calibration data\nprune_wanda(model, calib_data)\n\n# ❌ Bad: Too high sparsity in one shot\nprune(model, sparsity=0.9)  # Massive accuracy loss\n\n# ✅ Good: Gradual or iterative\niterative_prune(model, target=0.9, steps=10)\n```\n\n## Performance Comparison\n\n**Pruning methods at 50% sparsity** (LLaMA-7B):\n\n| Method | Accuracy Loss | Speed | Memory | Retraining Needed |\n|--------|---------------|-------|---------|-------------------|\n| **Magnitude** | -2.5% | 1.0× | -50% | No |\n| **Wanda** | -0.8% | 1.0× | -50% | No |\n| **SparseGPT** | -0.4% | 1.0× | -50% | No |\n| **N:M (2:4)** | -1.0% | 2.0× | -50% | No |\n| **Structured** | -3.0% | 2.0× | -50% | No |\n\n**Source**: Wanda paper (ICLR 2024), SparseGPT paper\n\n## Resources\n\n- **Wanda Paper (ICLR 2024)**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.11695\n- **Wanda GitHub**: https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\n- **SparseGPT Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.00774\n- **SparseGPT GitHub**: https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\n- **NVIDIA Sparse Tensor Cores**: https:\u002F\u002Fdeveloper.nvidia.com\u002Fblog\u002Faccelerating-inference-with-sparsity-using-ampere-and-tensorrt\u002F\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `awq-quantization`: Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss.\n- `quantizing-models-bitsandbytes`: Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss.\n- `speculative-decoding`: Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques.\n","# Model Pruning: Compressing LLMs\n\n## When to Use This Skill\n\nUse Model Pruning when you need to:\n- **Reduce model size** by 40-60% with \u003C1% accuracy loss\n- **Accelerate inference** using hardware-friendly sparsity (2-4× speedup)\n- **Deploy on constrained hardware** (mobile, edge devices)\n- **Compress without retraining** using one-shot methods\n- **Enable efficient serving** with reduced memory footprint\n\n**Key Techniques**: Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity\n\n**Papers**: Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774)\n\n## Installation\n\n```bash\n# Wanda implementation\ngit clone https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\ncd wanda\npip install -r requirements.txt\n\n# Optional: SparseGPT\ngit clone https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\ncd sparsegpt\npip install -e .\n\n# Dependencies\npip install torch transformers accelerate\n```\n\n## Quick Start\n\n### Wanda Pruning (One-Shot, No Retraining)\n\n**Source**: ICLR 2024 (arXiv 2306.11695)\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\u002FLlama-2-7b-hf\",\n    torch_dtype=torch.float16,\n    device_map=\"cuda\"\n)\ntokenizer = AutoTokenizer.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Calibration data (small dataset for activation statistics)\ncalib_data = [\n    \"The quick brown fox jumps over the lazy dog.\",\n    \"Machine learning is transforming the world.\",\n    \"Artificial intelligence powers modern applications.\",\n]\n\n# Wanda pruning function\ndef wanda_prune(model, calib_data, sparsity=0.5):\n    \"\"\"\n    Wanda: Prune by weight magnitude × input activation.\n\n    Args:\n        sparsity: Fraction of weights to prune (0.5 = 50%)\n    \"\"\"\n    # 1. Collect activation statistics\n    activations = {}\n\n    def hook_fn(name):\n        def hook(module, input, output):\n            # Store input activation norms\n            activations[name] = input[0].detach().abs().mean(dim=0)\n        return hook\n\n    # Register hooks for all linear layers\n    hooks = []\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            hooks.append(module.register_forward_hook(hook_fn(name)))\n\n    # Run calibration data\n    model.eval()\n    with torch.no_grad():\n        for text in calib_data:\n            inputs = tokenizer(text, return_tensors=\"pt\").to(model.device)\n            model(**inputs)\n\n    # Remove hooks\n    for hook in hooks:\n        hook.remove()\n\n    # 2. Prune weights based on |weight| × activation\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear) and name in activations:\n            W = module.weight.data\n            act = activations[name]\n\n            # Compute importance: |weight| × activation\n            importance = W.abs() * act.unsqueeze(0)\n\n            # Flatten and find threshold\n            threshold = torch.quantile(importance.flatten(), sparsity)\n\n            # Create mask\n            mask = importance >= threshold\n\n            # Apply mask (prune)\n            W *= mask.float()\n\n    return model\n\n# Apply Wanda pruning (50% sparsity, one-shot, no retraining)\npruned_model = wanda_prune(model, calib_data, sparsity=0.5)\n\n# Save\npruned_model.save_pretrained(\".\u002Fllama-2-7b-wanda-50\")\n```\n\n### SparseGPT (Second-Order Pruning)\n\n**Source**: arXiv 2301.00774\n\n```python\nfrom sparsegpt import SparseGPT\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Initialize SparseGPT\npruner = SparseGPT(model)\n\n# Calibration data\ncalib_data = load_calibration_data()  # ~128 samples\n\n# Prune (one-shot, layer-wise reconstruction)\npruned_model = pruner.prune(\n    calib_data=calib_data,\n    sparsity=0.5,           # 50% sparsity\n    prunen=0,               # Unstructured (0) or N:M structured\n    prunem=0,\n    percdamp=0.01,          # Damping for Hessian inverse\n)\n\n# Results: Near-lossless pruning at 50% sparsity\n```\n\n### N:M Structured Pruning (Hardware Accelerator)\n\n```python\ndef nm_prune(weight, n=2, m=4):\n    \"\"\"\n    N:M pruning: Keep N weights per M consecutive weights.\n    Example: 2:4 = keep 2 out of every 4 weights.\n\n    Compatible with NVIDIA sparse tensor cores (2:4, 4:8).\n    \"\"\"\n    # Reshape weight into groups of M\n    shape = weight.shape\n    weight_flat = weight.flatten()\n\n    # Pad to multiple of M\n    pad_size = (m - weight_flat.numel() % m) % m\n    weight_padded = F.pad(weight_flat, (0, pad_size))\n\n    # Reshape into (num_groups, m)\n    weight_grouped = weight_padded.reshape(-1, m)\n\n    # Find top-N in each group\n    _, indices = torch.topk(weight_grouped.abs(), n, dim=-1)\n\n    # Create mask\n    mask = torch.zeros_like(weight_grouped)\n    mask.scatter_(1, indices, 1.0)\n\n    # Apply mask\n    weight_pruned = weight_grouped * mask\n\n    # Reshape back\n    weight_pruned = weight_pruned.flatten()[:weight_flat.numel()]\n    return weight_pruned.reshape(shape)\n\n# Apply 2:4 sparsity (NVIDIA hardware)\nfor name, module in model.named_modules():\n    if isinstance(module, torch.nn.Linear):\n        module.weight.data = nm_prune(module.weight.data, n=2, m=4)\n\n# 50% sparsity, 2× speedup on A100 with sparse tensor cores\n```\n\n## Core Concepts\n\n### 1. Pruning Criteria\n\n**Magnitude Pruning** (baseline):\n```python\n# Prune weights with smallest absolute values\nimportance = weight.abs()\nthreshold = torch.quantile(importance, sparsity)\nmask = importance >= threshold\n```\n\n**Wanda** (weights × activations):\n```python\n# Importance = |weight| × input_activation\nimportance = weight.abs() * activation\n# Better than magnitude alone (considers usage)\n```\n\n**SparseGPT** (second-order):\n```python\n# Uses Hessian (second derivative) for importance\n# More accurate but computationally expensive\nimportance = weight^2 \u002F diag(Hessian)\n```\n\n### 2. Structured vs Unstructured\n\n**Unstructured** (fine-grained):\n- Prune individual weights\n- Higher quality (better accuracy)\n- No hardware speedup (irregular sparsity)\n\n**Structured** (coarse-grained):\n- Prune entire neurons, heads, or layers\n- Lower quality (more accuracy loss)\n- Hardware speedup (regular sparsity)\n\n**Semi-structured (N:M)**:\n- Best of both worlds\n- 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs\n- Minimal accuracy loss\n\n### 3. Sparsity Patterns\n\n```python\n# Unstructured (random)\n# [1, 0, 1, 0, 1, 1, 0, 0]\n# Pros: Flexible, high quality\n# Cons: No speedup\n\n# Structured (block)\n# [1, 1, 0, 0, 1, 1, 0, 0]\n# Pros: Hardware friendly\n# Cons: More accuracy loss\n\n# N:M (semi-structured)\n# [1, 0, 1, 0] [1, 1, 0, 0]  (2:4 pattern)\n# Pros: Hardware speedup + good quality\n# Cons: Requires specific hardware (NVIDIA)\n```\n\n## Pruning Strategies\n\n### Strategy 1: Gradual Magnitude Pruning\n\n```python\ndef gradual_prune(model, initial_sparsity=0.0, final_sparsity=0.5, num_steps=100):\n    \"\"\"Gradually increase sparsity during training.\"\"\"\n    for step in range(num_steps):\n        # Current sparsity\n        current_sparsity = initial_sparsity + (final_sparsity - initial_sparsity) * (step \u002F num_steps)\n\n        # Prune at current sparsity\n        for module in model.modules():\n            if isinstance(module, torch.nn.Linear):\n                weight = module.weight.data\n                threshold = torch.quantile(weight.abs().flatten(), current_sparsity)\n                mask = weight.abs() >= threshold\n                weight *= mask.float()\n\n        # Train one step\n        train_step(model)\n\n    return model\n```\n\n### Strategy 2: Layer-wise Pruning\n\n```python\ndef layer_wise_prune(model, sparsity_per_layer):\n    \"\"\"Different sparsity for different layers.\"\"\"\n    # Early layers: Less pruning (more important)\n    # Late layers: More pruning (less critical)\n\n    sparsity_schedule = {\n        \"layer.0\": 0.3,   # 30% sparsity\n        \"layer.1\": 0.4,\n        \"layer.2\": 0.5,\n        \"layer.3\": 0.6,   # 60% sparsity\n    }\n\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            # Find layer index\n            for layer_name, sparsity in sparsity_schedule.items():\n                if layer_name in name:\n                    # Prune at layer-specific sparsity\n                    prune_layer(module, sparsity)\n                    break\n\n    return model\n```\n\n### Strategy 3: Iterative Pruning + Fine-tuning\n\n```python\ndef iterative_prune_finetune(model, target_sparsity=0.5, iterations=5):\n    \"\"\"Prune gradually with fine-tuning between iterations.\"\"\"\n    current_sparsity = 0.0\n    sparsity_increment = target_sparsity \u002F iterations\n\n    for i in range(iterations):\n        # Increase sparsity\n        current_sparsity += sparsity_increment\n\n        # Prune\n        prune_model(model, sparsity=current_sparsity)\n\n        # Fine-tune (recover accuracy)\n        fine_tune(model, epochs=2, lr=1e-5)\n\n    return model\n\n# Results: Better accuracy than one-shot at high sparsity\n```\n\n## Production Deployment\n\nDetails, code examples and parameter tables: [references\u002Fproduction-deployment.md](references\u002Fproduction-deployment.md). Read it when this step applies.\n\n## Best Practices\n\n### 1. Sparsity Selection\n\n```python\n# Conservative (safe)\nsparsity = 0.3  # 30%, \u003C0.5% loss\n\n# Balanced (recommended)\nsparsity = 0.5  # 50%, ~1% loss\n\n# Aggressive (risky)\nsparsity = 0.7  # 70%, 2-5% loss\n\n# Extreme (model-dependent)\nsparsity = 0.9  # 90%, significant degradation\n```\n\n### 2. Method Selection\n\n```python\n# One-shot, no retraining → Wanda or SparseGPT\nif no_retraining_budget:\n    use_method = \"wanda\"  # Faster\n\n# Best quality → SparseGPT\nif need_best_quality:\n    use_method = \"sparsegpt\"  # More accurate\n\n# Hardware speedup → N:M structured\nif need_speedup:\n    use_method = \"nm_prune\"  # 2:4 or 4:8\n```\n\n### 3. Avoid Common Pitfalls\n\n```python\n# ❌ Bad: Pruning without calibration data\nprune_random(model)  # No activation statistics\n\n# ✅ Good: Use calibration data\nprune_wanda(model, calib_data)\n\n# ❌ Bad: Too high sparsity in one shot\nprune(model, sparsity=0.9)  # Massive accuracy loss\n\n# ✅ Good: Gradual or iterative\niterative_prune(model, target=0.9, steps=10)\n```\n\n## Performance Comparison\n\n**Pruning methods at 50% sparsity** (LLaMA-7B):\n\n| Method | Accuracy Loss | Speed | Memory | Retraining Needed |\n|--------|---------------|-------|---------|-------------------|\n| **Magnitude** | -2.5% | 1.0× | -50% | No |\n| **Wanda** | -0.8% | 1.0× | -50% | No |\n| **SparseGPT** | -0.4% | 1.0× | -50% | No |\n| **N:M (2:4)** | -1.0% | 2.0× | -50% | No |\n| **Structured** | -3.0% | 2.0× | -50% | No |\n\n**Source**: Wanda paper (ICLR 2024), SparseGPT paper\n\n## Resources\n\n- **Wanda Paper (ICLR 2024)**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.11695\n- **Wanda GitHub**: https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\n- **SparseGPT Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.00774\n- **SparseGPT GitHub**: https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\n- **NVIDIA Sparse Tensor Cores**: https:\u002F\u002Fdeveloper.nvidia.com\u002Fblog\u002Faccelerating-inference-with-sparsity-using-ampere-and-tensorrt\u002F\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `awq-quantization`: Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss.\n- `quantizing-models-bitsandbytes`: Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss.\n- `speculative-decoding`: Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques.\n",{"name":5,"license":15,"metadata":24,"description":7},{"tags":25,"version":26,"category":27,"maintainer":28,"dependencies":29},"Emerging Techniques, Model Pruning, Wanda, SparseGPT, Sparsity, Model Compression, N:M Sparsity, One-Shot Pruning, Structured Pruning, Unstructured Pruning, Fast Inference","1.0.0","multimodal-and-emerging","Kalaris Labs","transformers, torch","main","29fe36d26127205057d5e0ed8425f6b0e9a8c13a","skills\u002Fmodel-pruning",16,"https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills","https:\u002F\u002Fdocs.kalarislabs.com\u002Fresearch-agent-skills","https:\u002F\u002Fgithub.com\u002FKalarisLabs","kalarislabs.com","Applied AI lab building the infrastructure for the agentic era","2026-11-14 17:49:08",{"source":8,"skillName":5,"commands":41,"prompt":51,"zipAvailable":52},[42,45,48],{"tool":43,"command":44},"npx","npx skills add KalarisLabs\u002Fresearch-agent-skills --skill model-pruning",{"tool":46,"command":47},"bunx","bunx skills add KalarisLabs\u002Fresearch-agent-skills --skill model-pruning",{"tool":49,"command":50},"pnpm","pnpm dlx skills add KalarisLabs\u002Fresearch-agent-skills --skill model-pruning","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fmodel-pruning\u002FSKILL.md",true,[54,60],{"path":55,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":58,"sourceHtmlUrl":59},"references\u002Fproduction-deployment.md","REFERENCE","参考文档",2299,"https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fmodel-pruning\u002Freferences\u002Fproduction-deployment.md",{"path":61,"resourceType":56,"resourceTypeDesc":57,"sizeBytes":62,"sourceHtmlUrl":63},"references\u002Fwanda.md",9492,"https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fmodel-pruning\u002Freferences\u002Fwanda.md",[65,72,79,86,93,100],{"skillNo":66,"name":67,"nameZh":6,"description":68,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":69,"categories":70,"keywords":71,"publishedAt":19},"01M74Z8MGT305R11SNY5G28QH5","plos","Prepare manuscripts for PLOS journals (PLOS ONE, PLOS Biology, PLOS Computational Biology, PLOS Genetics, PLOS Medicine, PLOS Pathogens, PLOS Neglected Tropical Diseases, PLOS Global Public Health and others), covering structure, abstract and author summary, Vancouver-style references, the mandatory data availability policy, ethics and financial disclosure statements, figure requirements (PACE), the PLOS LaTeX template, reporting guidelines and PLOS ONE's publication criteria. Use when targeting any PLOS journal or checking compliance with PLOS policies.","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fplos\u002FSKILL.md",[],[],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":19},"01M74Z8CCN9APXYHNQ95V1EBGT","pkpd-modeling","Pharmacokinetic and pharmacodynamic modelling and simulation - non-compartmental analysis, compartmental and population PK, PK\u002FPD and exposure-response, TMDD, PBPK orientation, bioequivalence, allometric scaling and first-in-human dose, drug interaction prediction, and Bayesian therapeutic drug monitoring. Use when analysing concentration-time data, deriving exposure metrics, fitting PK or PD models, or evaluating dosing regimens. Triggers include \"pharmacokinetics\", \"pharmacodynamics\", \"PK\u002FPD\", \"NCA\", \"non-compartmental\", \"AUC\", \"Cmax\", \"lambda z\", \"half-life\", \"clearance\", \"volume of distribution\", \"compartmental model\", \"population PK\", \"popPK\", \"NONMEM\", \"nlmixr2\", \"Pharmpy\", \"Monolix\", \"exposure-response\", \"Emax\", \"EC50\", \"indirect response\", \"effect compartment\", \"TMDD\", \"PBPK\", \"bioequivalence\", \"RSABE\", \"ABEL\", \"allometric scaling\", \"first-in-human\", \"MABEL\", \"drug-drug interaction\", \"DDI\", \"ICH M12\", \"concentration-QTc\", \"therapeutic drug monitoring\", \"MIPD\", and \"dosing regimen\".","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fpkpd-modeling\u002FSKILL.md",[],[],{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":19},"01M74Z86YHJJ1FM3X87BHN76KZ","pinecone","Guides use of Pinecone, a managed serverless vector database, through its Python client and the LangChain and LlamaIndex integrations. Covers creating indexes, upserting and querying vectors, metadata filtering, namespaces, hybrid dense and sparse search, index management, and deleting vectors. Use when building a production RAG system on a hosted vector store, adding semantic search or recommendations without running infrastructure, isolating per-user or per-tenant data with namespaces, combining dense and sparse vectors in one query, or filtering results by metadata. Do not use for self-hosted or local stores (Chroma, Weaviate) or offline similarity search (FAISS).","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fpinecone\u002FSKILL.md",[],[],{"skillNo":87,"name":88,"nameZh":6,"description":89,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":19},"01M74Z80T6E89SAJ8MYDD0767Q","pi-agent","Build with and use Pi, the minimal terminal coding harness. Use for installing Pi, configuring providers\u002Fmodels\u002Fsettings\u002Fenvironment variables, creating Pi skills\u002Fextensions\u002Fpackages\u002Fthemes\u002Fprompt templates, embedding Pi through the SDK, integrating over RPC or JSON event streams, parsing sessions, running local models through the llama.cpp router, developing custom Pi providers and TUI components, or using ecosystem packages such as pi-subagents (delegation\u002Forchestration), pi-mcp-adapter (MCP servers), pi-interview (interactive forms), and pi-web-access (web search, fetching, video understanding).","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fpi-agent\u002FSKILL.md",[],[],{"skillNo":94,"name":95,"nameZh":6,"description":96,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":97,"categories":98,"keywords":99,"publishedAt":19},"01M74Z7WF1Y8Q3VFAQPPPRAFMV","phoenix-observability","Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fphoenix-observability\u002FSKILL.md",[],[],{"skillNo":101,"name":102,"nameZh":6,"description":103,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":104,"categories":105,"keywords":106,"publishedAt":19},"01M74Z7HNEH1A2AVYR3G1X2QV6","peft-fine-tuning","Fine-tunes LLMs with Hugging Face PEFT, using LoRA, QLoRA, IA3, AdaLoRA, prefix tuning, and prompt tuning so that under 1% of parameters are trained. Covers rank, alpha, and target module selection, loading and merging adapters, multi-adapter serving, and integration with TRL SFTTrainer, Axolotl, and vLLM. Use when fine-tuning 7B-70B models on limited GPU memory, when running QLoRA on a single 24GB GPU, when choosing LoRA rank and alpha, when merging or swapping adapters on one base model, or when debugging CUDA OOM or an adapter that does not apply. Not for full fine-tuning of small models under 1B parameters or cases needing all weights updated.","https:\u002F\u002Fgithub.com\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fblob\u002Fmain\u002Fskills\u002Fpeft-fine-tuning\u002FSKILL.md",[],[]]