[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6FC2W2R07T1ZNPWMMYSB7AQ":3,"skill-same:01M6FC2W2R07T1ZNPWMMYSB7AQ":84},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":55,"commitSha":56,"skillDirectory":57,"forksCount":58,"repositoryHtmlUrl":59,"repositoryHomepageUrl":6,"ownerHtmlUrl":60,"ownerBlogUrl":61,"ownerDescription":62,"updatedAt":63,"install":64,"resources":77},"01M6FC2W2R07T1ZNPWMMYSB7AQ","model-pruning",null,"Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.","Dicklesworthstone\u002Fpi_agent_rust","Dicklesworthstone","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F35050222?v=4","01M6CG26AR942CSVMN1M5E3DSE","Jeff Emanuel","01M6CG26EPP71PSHW1VDS71121",1991,"MIT","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Femerging-techniques-model-pruning\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"model","Model Pruning","Wanda","when","accuracy","reduce","orchestra","unstructured","with","Unstructured Pruning","Structured Pruning","One-Shot Pruning","N:M Sparsity","Sparsity","SparseGPT","research","Emerging Techniques","Model Compression","Accelerate","methods","models","minimal","use","Transformers","Fast Inference","2026-11-10 10:12:30",false,"---\nname: model-pruning\ndescription: Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Emerging Techniques, Model Pruning, Wanda, SparseGPT, Sparsity, Model Compression, N:M Sparsity, One-Shot Pruning, Structured Pruning, Unstructured Pruning, Fast Inference]\ndependencies: [transformers, torch]\n---\n\n# Model Pruning: Compressing LLMs\n\n## When to Use This Skill\n\nUse Model Pruning when you need to:\n- **Reduce model size** by 40-60% with \u003C1% accuracy loss\n- **Accelerate inference** using hardware-friendly sparsity (2-4× speedup)\n- **Deploy on constrained hardware** (mobile, edge devices)\n- **Compress without retraining** using one-shot methods\n- **Enable efficient serving** with reduced memory footprint\n\n**Key Techniques**: Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity\n\n**Papers**: Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774)\n\n## Installation\n\n```bash\n# Wanda implementation\ngit clone https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\ncd wanda\npip install -r requirements.txt\n\n# Optional: SparseGPT\ngit clone https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\ncd sparsegpt\npip install -e .\n\n# Dependencies\npip install torch transformers accelerate\n```\n\n## Quick Start\n\n### Wanda Pruning (One-Shot, No Retraining)\n\n**Source**: ICLR 2024 (arXiv 2306.11695)\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\u002FLlama-2-7b-hf\",\n    torch_dtype=torch.float16,\n    device_map=\"cuda\"\n)\ntokenizer = AutoTokenizer.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Calibration data (small dataset for activation statistics)\ncalib_data = [\n    \"The quick brown fox jumps over the lazy dog.\",\n    \"Machine learning is transforming the world.\",\n    \"Artificial intelligence powers modern applications.\",\n]\n\n# Wanda pruning function\ndef wanda_prune(model, calib_data, sparsity=0.5):\n    \"\"\"\n    Wanda: Prune by weight magnitude × input activation.\n\n    Args:\n        sparsity: Fraction of weights to prune (0.5 = 50%)\n    \"\"\"\n    # 1. Collect activation statistics\n    activations = {}\n\n    def hook_fn(name):\n        def hook(module, input, output):\n            # Store input activation norms\n            activations[name] = input[0].detach().abs().mean(dim=0)\n        return hook\n\n    # Register hooks for all linear layers\n    hooks = []\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            hooks.append(module.register_forward_hook(hook_fn(name)))\n\n    # Run calibration data\n    model.eval()\n    with torch.no_grad():\n        for text in calib_data:\n            inputs = tokenizer(text, return_tensors=\"pt\").to(model.device)\n            model(**inputs)\n\n    # Remove hooks\n    for hook in hooks:\n        hook.remove()\n\n    # 2. Prune weights based on |weight| × activation\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear) and name in activations:\n            W = module.weight.data\n            act = activations[name]\n\n            # Compute importance: |weight| × activation\n            importance = W.abs() * act.unsqueeze(0)\n\n            # Flatten and find threshold\n            threshold = torch.quantile(importance.flatten(), sparsity)\n\n            # Create mask\n            mask = importance >= threshold\n\n            # Apply mask (prune)\n            W *= mask.float()\n\n    return model\n\n# Apply Wanda pruning (50% sparsity, one-shot, no retraining)\npruned_model = wanda_prune(model, calib_data, sparsity=0.5)\n\n# Save\npruned_model.save_pretrained(\".\u002Fllama-2-7b-wanda-50\")\n```\n\n### SparseGPT (Second-Order Pruning)\n\n**Source**: arXiv 2301.00774\n\n```python\nfrom sparsegpt import SparseGPT\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Initialize SparseGPT\npruner = SparseGPT(model)\n\n# Calibration data\ncalib_data = load_calibration_data()  # ~128 samples\n\n# Prune (one-shot, layer-wise reconstruction)\npruned_model = pruner.prune(\n    calib_data=calib_data,\n    sparsity=0.5,           # 50% sparsity\n    prunen=0,               # Unstructured (0) or N:M structured\n    prunem=0,\n    percdamp=0.01,          # Damping for Hessian inverse\n)\n\n# Results: Near-lossless pruning at 50% sparsity\n```\n\n### N:M Structured Pruning (Hardware Accelerator)\n\n```python\ndef nm_prune(weight, n=2, m=4):\n    \"\"\"\n    N:M pruning: Keep N weights per M consecutive weights.\n    Example: 2:4 = keep 2 out of every 4 weights.\n\n    Compatible with NVIDIA sparse tensor cores (2:4, 4:8).\n    \"\"\"\n    # Reshape weight into groups of M\n    shape = weight.shape\n    weight_flat = weight.flatten()\n\n    # Pad to multiple of M\n    pad_size = (m - weight_flat.numel() % m) % m\n    weight_padded = F.pad(weight_flat, (0, pad_size))\n\n    # Reshape into (num_groups, m)\n    weight_grouped = weight_padded.reshape(-1, m)\n\n    # Find top-N in each group\n    _, indices = torch.topk(weight_grouped.abs(), n, dim=-1)\n\n    # Create mask\n    mask = torch.zeros_like(weight_grouped)\n    mask.scatter_(1, indices, 1.0)\n\n    # Apply mask\n    weight_pruned = weight_grouped * mask\n\n    # Reshape back\n    weight_pruned = weight_pruned.flatten()[:weight_flat.numel()]\n    return weight_pruned.reshape(shape)\n\n# Apply 2:4 sparsity (NVIDIA hardware)\nfor name, module in model.named_modules():\n    if isinstance(module, torch.nn.Linear):\n        module.weight.data = nm_prune(module.weight.data, n=2, m=4)\n\n# 50% sparsity, 2× speedup on A100 with sparse tensor cores\n```\n\n## Core Concepts\n\n### 1. Pruning Criteria\n\n**Magnitude Pruning** (baseline):\n```python\n# Prune weights with smallest absolute values\nimportance = weight.abs()\nthreshold = torch.quantile(importance, sparsity)\nmask = importance >= threshold\n```\n\n**Wanda** (weights × activations):\n```python\n# Importance = |weight| × input_activation\nimportance = weight.abs() * activation\n# Better than magnitude alone (considers usage)\n```\n\n**SparseGPT** (second-order):\n```python\n# Uses Hessian (second derivative) for importance\n# More accurate but computationally expensive\nimportance = weight^2 \u002F diag(Hessian)\n```\n\n### 2. Structured vs Unstructured\n\n**Unstructured** (fine-grained):\n- Prune individual weights\n- Higher quality (better accuracy)\n- No hardware speedup (irregular sparsity)\n\n**Structured** (coarse-grained):\n- Prune entire neurons, heads, or layers\n- Lower quality (more accuracy loss)\n- Hardware speedup (regular sparsity)\n\n**Semi-structured (N:M)**:\n- Best of both worlds\n- 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs\n- Minimal accuracy loss\n\n### 3. Sparsity Patterns\n\n```python\n# Unstructured (random)\n# [1, 0, 1, 0, 1, 1, 0, 0]\n# Pros: Flexible, high quality\n# Cons: No speedup\n\n# Structured (block)\n# [1, 1, 0, 0, 1, 1, 0, 0]\n# Pros: Hardware friendly\n# Cons: More accuracy loss\n\n# N:M (semi-structured)\n# [1, 0, 1, 0] [1, 1, 0, 0]  (2:4 pattern)\n# Pros: Hardware speedup + good quality\n# Cons: Requires specific hardware (NVIDIA)\n```\n\n## Pruning Strategies\n\n### Strategy 1: Gradual Magnitude Pruning\n\n```python\ndef gradual_prune(model, initial_sparsity=0.0, final_sparsity=0.5, num_steps=100):\n    \"\"\"Gradually increase sparsity during training.\"\"\"\n    for step in range(num_steps):\n        # Current sparsity\n        current_sparsity = initial_sparsity + (final_sparsity - initial_sparsity) * (step \u002F num_steps)\n\n        # Prune at current sparsity\n        for module in model.modules():\n            if isinstance(module, torch.nn.Linear):\n                weight = module.weight.data\n                threshold = torch.quantile(weight.abs().flatten(), current_sparsity)\n                mask = weight.abs() >= threshold\n                weight *= mask.float()\n\n        # Train one step\n        train_step(model)\n\n    return model\n```\n\n### Strategy 2: Layer-wise Pruning\n\n```python\ndef layer_wise_prune(model, sparsity_per_layer):\n    \"\"\"Different sparsity for different layers.\"\"\"\n    # Early layers: Less pruning (more important)\n    # Late layers: More pruning (less critical)\n\n    sparsity_schedule = {\n        \"layer.0\": 0.3,   # 30% sparsity\n        \"layer.1\": 0.4,\n        \"layer.2\": 0.5,\n        \"layer.3\": 0.6,   # 60% sparsity\n    }\n\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            # Find layer index\n            for layer_name, sparsity in sparsity_schedule.items():\n                if layer_name in name:\n                    # Prune at layer-specific sparsity\n                    prune_layer(module, sparsity)\n                    break\n\n    return model\n```\n\n### Strategy 3: Iterative Pruning + Fine-tuning\n\n```python\ndef iterative_prune_finetune(model, target_sparsity=0.5, iterations=5):\n    \"\"\"Prune gradually with fine-tuning between iterations.\"\"\"\n    current_sparsity = 0.0\n    sparsity_increment = target_sparsity \u002F iterations\n\n    for i in range(iterations):\n        # Increase sparsity\n        current_sparsity += sparsity_increment\n\n        # Prune\n        prune_model(model, sparsity=current_sparsity)\n\n        # Fine-tune (recover accuracy)\n        fine_tune(model, epochs=2, lr=1e-5)\n\n    return model\n\n# Results: Better accuracy than one-shot at high sparsity\n```\n\n## Production Deployment\n\n### Complete Pruning Pipeline\n\n```python\nfrom transformers import Trainer, TrainingArguments\n\ndef production_pruning_pipeline(\n    model_name=\"meta-llama\u002FLlama-2-7b-hf\",\n    target_sparsity=0.5,\n    method=\"wanda\",  # or \"sparsegpt\"\n):\n    # 1. Load model\n    model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)\n    tokenizer = AutoTokenizer.from_pretrained(model_name)\n\n    # 2. Load calibration data\n    calib_dataset = load_dataset(\"wikitext\", \"wikitext-2-raw-v1\", split=\"train[:1000]\")\n\n    # 3. Apply pruning\n    if method == \"wanda\":\n        pruned_model = wanda_prune(model, calib_dataset, sparsity=target_sparsity)\n    elif method == \"sparsegpt\":\n        pruner = SparseGPT(model)\n        pruned_model = pruner.prune(calib_dataset, sparsity=target_sparsity)\n\n    # 4. (Optional) Fine-tune to recover accuracy\n    training_args = TrainingArguments(\n        output_dir=\".\u002Fpruned-model\",\n        num_train_epochs=1,\n        per_device_train_batch_size=4,\n        learning_rate=1e-5,\n        bf16=True,\n    )\n\n    trainer = Trainer(\n        model=pruned_model,\n        args=training_args,\n        train_dataset=finetune_dataset,\n    )\n\n    trainer.train()\n\n    # 5. Save\n    pruned_model.save_pretrained(\".\u002Fpruned-llama-7b-50\")\n    tokenizer.save_pretrained(\".\u002Fpruned-llama-7b-50\")\n\n    return pruned_model\n\n# Usage\npruned_model = production_pruning_pipeline(\n    model_name=\"meta-llama\u002FLlama-2-7b-hf\",\n    target_sparsity=0.5,\n    method=\"wanda\"\n)\n```\n\n### Evaluation\n\n```python\nfrom lm_eval import evaluator\n\n# Evaluate pruned vs original model\noriginal_results = evaluator.simple_evaluate(\n    model=\"hf\",\n    model_args=\"pretrained=meta-llama\u002FLlama-2-7b-hf\",\n    tasks=[\"arc_easy\", \"hellaswag\", \"winogrande\"],\n)\n\npruned_results = evaluator.simple_evaluate(\n    model=\"hf\",\n    model_args=\"pretrained=.\u002Fpruned-llama-7b-50\",\n    tasks=[\"arc_easy\", \"hellaswag\", \"winogrande\"],\n)\n\n# Compare\nprint(f\"Original: {original_results['results']['arc_easy']['acc']:.3f}\")\nprint(f\"Pruned:   {pruned_results['results']['arc_easy']['acc']:.3f}\")\nprint(f\"Degradation: {(original_results - pruned_results):.3f}\")\n\n# Typical results at 50% sparsity:\n# - Wanda: \u003C1% accuracy loss\n# - SparseGPT: \u003C0.5% accuracy loss\n# - Magnitude: 2-3% accuracy loss\n```\n\n## Best Practices\n\n### 1. Sparsity Selection\n\n```python\n# Conservative (safe)\nsparsity = 0.3  # 30%, \u003C0.5% loss\n\n# Balanced (recommended)\nsparsity = 0.5  # 50%, ~1% loss\n\n# Aggressive (risky)\nsparsity = 0.7  # 70%, 2-5% loss\n\n# Extreme (model-dependent)\nsparsity = 0.9  # 90%, significant degradation\n```\n\n### 2. Method Selection\n\n```python\n# One-shot, no retraining → Wanda or SparseGPT\nif no_retraining_budget:\n    use_method = \"wanda\"  # Faster\n\n# Best quality → SparseGPT\nif need_best_quality:\n    use_method = \"sparsegpt\"  # More accurate\n\n# Hardware speedup → N:M structured\nif need_speedup:\n    use_method = \"nm_prune\"  # 2:4 or 4:8\n```\n\n### 3. Avoid Common Pitfalls\n\n```python\n# ❌ Bad: Pruning without calibration data\nprune_random(model)  # No activation statistics\n\n# ✅ Good: Use calibration data\nprune_wanda(model, calib_data)\n\n# ❌ Bad: Too high sparsity in one shot\nprune(model, sparsity=0.9)  # Massive accuracy loss\n\n# ✅ Good: Gradual or iterative\niterative_prune(model, target=0.9, steps=10)\n```\n\n## Performance Comparison\n\n**Pruning methods at 50% sparsity** (LLaMA-7B):\n\n| Method | Accuracy Loss | Speed | Memory | Retraining Needed |\n|--------|---------------|-------|---------|-------------------|\n| **Magnitude** | -2.5% | 1.0× | -50% | No |\n| **Wanda** | -0.8% | 1.0× | -50% | No |\n| **SparseGPT** | -0.4% | 1.0× | -50% | No |\n| **N:M (2:4)** | -1.0% | 2.0× | -50% | No |\n| **Structured** | -3.0% | 2.0× | -50% | No |\n\n**Source**: Wanda paper (ICLR 2024), SparseGPT paper\n\n## Resources\n\n- **Wanda Paper (ICLR 2024)**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.11695\n- **Wanda GitHub**: https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\n- **SparseGPT Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.00774\n- **SparseGPT GitHub**: https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\n- **NVIDIA Sparse Tensor Cores**: https:\u002F\u002Fdeveloper.nvidia.com\u002Fblog\u002Faccelerating-inference-with-sparsity-using-ampere-and-tensorrt\u002F\n\n\n","# Model Pruning: Compressing LLMs\n\n## When to Use This Skill\n\nUse Model Pruning when you need to:\n- **Reduce model size** by 40-60% with \u003C1% accuracy loss\n- **Accelerate inference** using hardware-friendly sparsity (2-4× speedup)\n- **Deploy on constrained hardware** (mobile, edge devices)\n- **Compress without retraining** using one-shot methods\n- **Enable efficient serving** with reduced memory footprint\n\n**Key Techniques**: Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity\n\n**Papers**: Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774)\n\n## Installation\n\n```bash\n# Wanda implementation\ngit clone https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\ncd wanda\npip install -r requirements.txt\n\n# Optional: SparseGPT\ngit clone https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\ncd sparsegpt\npip install -e .\n\n# Dependencies\npip install torch transformers accelerate\n```\n\n## Quick Start\n\n### Wanda Pruning (One-Shot, No Retraining)\n\n**Source**: ICLR 2024 (arXiv 2306.11695)\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\u002FLlama-2-7b-hf\",\n    torch_dtype=torch.float16,\n    device_map=\"cuda\"\n)\ntokenizer = AutoTokenizer.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Calibration data (small dataset for activation statistics)\ncalib_data = [\n    \"The quick brown fox jumps over the lazy dog.\",\n    \"Machine learning is transforming the world.\",\n    \"Artificial intelligence powers modern applications.\",\n]\n\n# Wanda pruning function\ndef wanda_prune(model, calib_data, sparsity=0.5):\n    \"\"\"\n    Wanda: Prune by weight magnitude × input activation.\n\n    Args:\n        sparsity: Fraction of weights to prune (0.5 = 50%)\n    \"\"\"\n    # 1. Collect activation statistics\n    activations = {}\n\n    def hook_fn(name):\n        def hook(module, input, output):\n            # Store input activation norms\n            activations[name] = input[0].detach().abs().mean(dim=0)\n        return hook\n\n    # Register hooks for all linear layers\n    hooks = []\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            hooks.append(module.register_forward_hook(hook_fn(name)))\n\n    # Run calibration data\n    model.eval()\n    with torch.no_grad():\n        for text in calib_data:\n            inputs = tokenizer(text, return_tensors=\"pt\").to(model.device)\n            model(**inputs)\n\n    # Remove hooks\n    for hook in hooks:\n        hook.remove()\n\n    # 2. Prune weights based on |weight| × activation\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear) and name in activations:\n            W = module.weight.data\n            act = activations[name]\n\n            # Compute importance: |weight| × activation\n            importance = W.abs() * act.unsqueeze(0)\n\n            # Flatten and find threshold\n            threshold = torch.quantile(importance.flatten(), sparsity)\n\n            # Create mask\n            mask = importance >= threshold\n\n            # Apply mask (prune)\n            W *= mask.float()\n\n    return model\n\n# Apply Wanda pruning (50% sparsity, one-shot, no retraining)\npruned_model = wanda_prune(model, calib_data, sparsity=0.5)\n\n# Save\npruned_model.save_pretrained(\".\u002Fllama-2-7b-wanda-50\")\n```\n\n### SparseGPT (Second-Order Pruning)\n\n**Source**: arXiv 2301.00774\n\n```python\nfrom sparsegpt import SparseGPT\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Initialize SparseGPT\npruner = SparseGPT(model)\n\n# Calibration data\ncalib_data = load_calibration_data()  # ~128 samples\n\n# Prune (one-shot, layer-wise reconstruction)\npruned_model = pruner.prune(\n    calib_data=calib_data,\n    sparsity=0.5,           # 50% sparsity\n    prunen=0,               # Unstructured (0) or N:M structured\n    prunem=0,\n    percdamp=0.01,          # Damping for Hessian inverse\n)\n\n# Results: Near-lossless pruning at 50% sparsity\n```\n\n### N:M Structured Pruning (Hardware Accelerator)\n\n```python\ndef nm_prune(weight, n=2, m=4):\n    \"\"\"\n    N:M pruning: Keep N weights per M consecutive weights.\n    Example: 2:4 = keep 2 out of every 4 weights.\n\n    Compatible with NVIDIA sparse tensor cores (2:4, 4:8).\n    \"\"\"\n    # Reshape weight into groups of M\n    shape = weight.shape\n    weight_flat = weight.flatten()\n\n    # Pad to multiple of M\n    pad_size = (m - weight_flat.numel() % m) % m\n    weight_padded = F.pad(weight_flat, (0, pad_size))\n\n    # Reshape into (num_groups, m)\n    weight_grouped = weight_padded.reshape(-1, m)\n\n    # Find top-N in each group\n    _, indices = torch.topk(weight_grouped.abs(), n, dim=-1)\n\n    # Create mask\n    mask = torch.zeros_like(weight_grouped)\n    mask.scatter_(1, indices, 1.0)\n\n    # Apply mask\n    weight_pruned = weight_grouped * mask\n\n    # Reshape back\n    weight_pruned = weight_pruned.flatten()[:weight_flat.numel()]\n    return weight_pruned.reshape(shape)\n\n# Apply 2:4 sparsity (NVIDIA hardware)\nfor name, module in model.named_modules():\n    if isinstance(module, torch.nn.Linear):\n        module.weight.data = nm_prune(module.weight.data, n=2, m=4)\n\n# 50% sparsity, 2× speedup on A100 with sparse tensor cores\n```\n\n## Core Concepts\n\n### 1. Pruning Criteria\n\n**Magnitude Pruning** (baseline):\n```python\n# Prune weights with smallest absolute values\nimportance = weight.abs()\nthreshold = torch.quantile(importance, sparsity)\nmask = importance >= threshold\n```\n\n**Wanda** (weights × activations):\n```python\n# Importance = |weight| × input_activation\nimportance = weight.abs() * activation\n# Better than magnitude alone (considers usage)\n```\n\n**SparseGPT** (second-order):\n```python\n# Uses Hessian (second derivative) for importance\n# More accurate but computationally expensive\nimportance = weight^2 \u002F diag(Hessian)\n```\n\n### 2. Structured vs Unstructured\n\n**Unstructured** (fine-grained):\n- Prune individual weights\n- Higher quality (better accuracy)\n- No hardware speedup (irregular sparsity)\n\n**Structured** (coarse-grained):\n- Prune entire neurons, heads, or layers\n- Lower quality (more accuracy loss)\n- Hardware speedup (regular sparsity)\n\n**Semi-structured (N:M)**:\n- Best of both worlds\n- 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs\n- Minimal accuracy loss\n\n### 3. Sparsity Patterns\n\n```python\n# Unstructured (random)\n# [1, 0, 1, 0, 1, 1, 0, 0]\n# Pros: Flexible, high quality\n# Cons: No speedup\n\n# Structured (block)\n# [1, 1, 0, 0, 1, 1, 0, 0]\n# Pros: Hardware friendly\n# Cons: More accuracy loss\n\n# N:M (semi-structured)\n# [1, 0, 1, 0] [1, 1, 0, 0]  (2:4 pattern)\n# Pros: Hardware speedup + good quality\n# Cons: Requires specific hardware (NVIDIA)\n```\n\n## Pruning Strategies\n\n### Strategy 1: Gradual Magnitude Pruning\n\n```python\ndef gradual_prune(model, initial_sparsity=0.0, final_sparsity=0.5, num_steps=100):\n    \"\"\"Gradually increase sparsity during training.\"\"\"\n    for step in range(num_steps):\n        # Current sparsity\n        current_sparsity = initial_sparsity + (final_sparsity - initial_sparsity) * (step \u002F num_steps)\n\n        # Prune at current sparsity\n        for module in model.modules():\n            if isinstance(module, torch.nn.Linear):\n                weight = module.weight.data\n                threshold = torch.quantile(weight.abs().flatten(), current_sparsity)\n                mask = weight.abs() >= threshold\n                weight *= mask.float()\n\n        # Train one step\n        train_step(model)\n\n    return model\n```\n\n### Strategy 2: Layer-wise Pruning\n\n```python\ndef layer_wise_prune(model, sparsity_per_layer):\n    \"\"\"Different sparsity for different layers.\"\"\"\n    # Early layers: Less pruning (more important)\n    # Late layers: More pruning (less critical)\n\n    sparsity_schedule = {\n        \"layer.0\": 0.3,   # 30% sparsity\n        \"layer.1\": 0.4,\n        \"layer.2\": 0.5,\n        \"layer.3\": 0.6,   # 60% sparsity\n    }\n\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            # Find layer index\n            for layer_name, sparsity in sparsity_schedule.items():\n                if layer_name in name:\n                    # Prune at layer-specific sparsity\n                    prune_layer(module, sparsity)\n                    break\n\n    return model\n```\n\n### Strategy 3: Iterative Pruning + Fine-tuning\n\n```python\ndef iterative_prune_finetune(model, target_sparsity=0.5, iterations=5):\n    \"\"\"Prune gradually with fine-tuning between iterations.\"\"\"\n    current_sparsity = 0.0\n    sparsity_increment = target_sparsity \u002F iterations\n\n    for i in range(iterations):\n        # Increase sparsity\n        current_sparsity += sparsity_increment\n\n        # Prune\n        prune_model(model, sparsity=current_sparsity)\n\n        # Fine-tune (recover accuracy)\n        fine_tune(model, epochs=2, lr=1e-5)\n\n    return model\n\n# Results: Better accuracy than one-shot at high sparsity\n```\n\n## Production Deployment\n\n### Complete Pruning Pipeline\n\n```python\nfrom transformers import Trainer, TrainingArguments\n\ndef production_pruning_pipeline(\n    model_name=\"meta-llama\u002FLlama-2-7b-hf\",\n    target_sparsity=0.5,\n    method=\"wanda\",  # or \"sparsegpt\"\n):\n    # 1. Load model\n    model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)\n    tokenizer = AutoTokenizer.from_pretrained(model_name)\n\n    # 2. Load calibration data\n    calib_dataset = load_dataset(\"wikitext\", \"wikitext-2-raw-v1\", split=\"train[:1000]\")\n\n    # 3. Apply pruning\n    if method == \"wanda\":\n        pruned_model = wanda_prune(model, calib_dataset, sparsity=target_sparsity)\n    elif method == \"sparsegpt\":\n        pruner = SparseGPT(model)\n        pruned_model = pruner.prune(calib_dataset, sparsity=target_sparsity)\n\n    # 4. (Optional) Fine-tune to recover accuracy\n    training_args = TrainingArguments(\n        output_dir=\".\u002Fpruned-model\",\n        num_train_epochs=1,\n        per_device_train_batch_size=4,\n        learning_rate=1e-5,\n        bf16=True,\n    )\n\n    trainer = Trainer(\n        model=pruned_model,\n        args=training_args,\n        train_dataset=finetune_dataset,\n    )\n\n    trainer.train()\n\n    # 5. Save\n    pruned_model.save_pretrained(\".\u002Fpruned-llama-7b-50\")\n    tokenizer.save_pretrained(\".\u002Fpruned-llama-7b-50\")\n\n    return pruned_model\n\n# Usage\npruned_model = production_pruning_pipeline(\n    model_name=\"meta-llama\u002FLlama-2-7b-hf\",\n    target_sparsity=0.5,\n    method=\"wanda\"\n)\n```\n\n### Evaluation\n\n```python\nfrom lm_eval import evaluator\n\n# Evaluate pruned vs original model\noriginal_results = evaluator.simple_evaluate(\n    model=\"hf\",\n    model_args=\"pretrained=meta-llama\u002FLlama-2-7b-hf\",\n    tasks=[\"arc_easy\", \"hellaswag\", \"winogrande\"],\n)\n\npruned_results = evaluator.simple_evaluate(\n    model=\"hf\",\n    model_args=\"pretrained=.\u002Fpruned-llama-7b-50\",\n    tasks=[\"arc_easy\", \"hellaswag\", \"winogrande\"],\n)\n\n# Compare\nprint(f\"Original: {original_results['results']['arc_easy']['acc']:.3f}\")\nprint(f\"Pruned:   {pruned_results['results']['arc_easy']['acc']:.3f}\")\nprint(f\"Degradation: {(original_results - pruned_results):.3f}\")\n\n# Typical results at 50% sparsity:\n# - Wanda: \u003C1% accuracy loss\n# - SparseGPT: \u003C0.5% accuracy loss\n# - Magnitude: 2-3% accuracy loss\n```\n\n## Best Practices\n\n### 1. Sparsity Selection\n\n```python\n# Conservative (safe)\nsparsity = 0.3  # 30%, \u003C0.5% loss\n\n# Balanced (recommended)\nsparsity = 0.5  # 50%, ~1% loss\n\n# Aggressive (risky)\nsparsity = 0.7  # 70%, 2-5% loss\n\n# Extreme (model-dependent)\nsparsity = 0.9  # 90%, significant degradation\n```\n\n### 2. Method Selection\n\n```python\n# One-shot, no retraining → Wanda or SparseGPT\nif no_retraining_budget:\n    use_method = \"wanda\"  # Faster\n\n# Best quality → SparseGPT\nif need_best_quality:\n    use_method = \"sparsegpt\"  # More accurate\n\n# Hardware speedup → N:M structured\nif need_speedup:\n    use_method = \"nm_prune\"  # 2:4 or 4:8\n```\n\n### 3. Avoid Common Pitfalls\n\n```python\n# ❌ Bad: Pruning without calibration data\nprune_random(model)  # No activation statistics\n\n# ✅ Good: Use calibration data\nprune_wanda(model, calib_data)\n\n# ❌ Bad: Too high sparsity in one shot\nprune(model, sparsity=0.9)  # Massive accuracy loss\n\n# ✅ Good: Gradual or iterative\niterative_prune(model, target=0.9, steps=10)\n```\n\n## Performance Comparison\n\n**Pruning methods at 50% sparsity** (LLaMA-7B):\n\n| Method | Accuracy Loss | Speed | Memory | Retraining Needed |\n|--------|---------------|-------|---------|-------------------|\n| **Magnitude** | -2.5% | 1.0× | -50% | No |\n| **Wanda** | -0.8% | 1.0× | -50% | No |\n| **SparseGPT** | -0.4% | 1.0× | -50% | No |\n| **N:M (2:4)** | -1.0% | 2.0× | -50% | No |\n| **Structured** | -3.0% | 2.0× | -50% | No |\n\n**Source**: Wanda paper (ICLR 2024), SparseGPT paper\n\n## Resources\n\n- **Wanda Paper (ICLR 2024)**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.11695\n- **Wanda GitHub**: https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\n- **SparseGPT Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.00774\n- **SparseGPT GitHub**: https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\n- **NVIDIA Sparse Tensor Cores**: https:\u002F\u002Fdeveloper.nvidia.com\u002Fblog\u002Faccelerating-inference-with-sparsity-using-ampere-and-tensorrt\u002F\n\n\n",{"name":5,"tags":49,"author":50,"license":15,"version":51,"description":7,"dependencies":52},[35,20,21,33,32,36,31,30,29,28,43],"Orchestra Research","1.0.0",[53,54],"transformers","torch","main","90b50ed0778745a80fd5bfb13b1843d45b9e82f7","tests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Femerging-techniques-model-pruning",234,"https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust","https:\u002F\u002Fgithub.com\u002FDicklesworthstone","https:\u002F\u002Fwww.jeffreyemanuel.com\u002F","Building in NY","2026-11-13 11:50:27",{"source":8,"skillName":5,"commands":65,"prompt":75,"zipAvailable":76},[66,69,72],{"tool":67,"command":68},"npx","npx skills add Dicklesworthstone\u002Fpi_agent_rust --skill model-pruning",{"tool":70,"command":71},"bunx","bunx skills add Dicklesworthstone\u002Fpi_agent_rust --skill model-pruning",{"tool":73,"command":74},"pnpm","pnpm dlx skills add Dicklesworthstone\u002Fpi_agent_rust --skill model-pruning","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Femerging-techniques-model-pruning\u002FSKILL.md",true,[78],{"path":79,"resourceType":80,"resourceTypeDesc":81,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fwanda.md","REFERENCE","参考文档",9492,"https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Femerging-techniques-model-pruning\u002Freferences\u002Fwanda.md",[85,125,149,175,202,224],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":89,"licenseFileUrl":90,"sourceHtmlUrl":91,"categories":92,"keywords":98,"publishedAt":44},"01M6FEGPPK0ZTKCCM0R3E3KS33","web-artifacts-builder","Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn\u002Fui). Use for complex artifacts requiring state management, routing, or shadcn\u002Fui components - not for simple single-file HTML\u002FJSX artifacts.","Apache-2.0","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Futilities\u002Fweb-artifacts-builder\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Futilities\u002Fweb-artifacts-builder\u002FSKILL.md",[93],{"categoryNo":94,"categoryName":95,"taxonomyType":96,"taxonomyTypeDesc":97},"01JBSEEDCAT000000000000009","前端开发","SCENARIO","使用场景",[99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,41,117,118,119,120,121,122,123,124],"web artifacts builder","web","artifacts","artifacts builder","builder","frontend","css","components","react","component","routing","state management","state","shadcn","file","ui","Tools","Tailwind","html","single-file","simple","ui components","multi","for","jsx","suite",{"skillNo":126,"name":127,"nameZh":6,"description":128,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":89,"licenseFileUrl":129,"sourceHtmlUrl":130,"categories":131,"keywords":136,"publishedAt":44},"01M6FEE9VTK1W4GY21AKREJXRG","webapp-testing","Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fsecurity\u002Fwebapp-testing\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fsecurity\u002Fwebapp-testing\u002FSKILL.md",[132,135],{"categoryNo":133,"categoryName":134,"taxonomyType":96,"taxonomyTypeDesc":97},"01JBSEEDCAT000000000000008","测试与质量",{"categoryNo":94,"categoryName":95,"taxonomyType":96,"taxonomyTypeDesc":97},[137,138,139,104,140,141,100,142,143,144,114,145,146,147,148,27,122],"testing","webapp testing","webapp","screenshots","playwright","debugging","toolkit","browser","logs","applications","behavior","local",{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":153,"sourceHtmlUrl":154,"categories":155,"keywords":156,"publishedAt":44},"01M6FE20F7ZTQBFWKF4MPJ2SBX","neuropixels-analysis","Neuropixels neural recording analysis. Load SpikeGLX\u002FOpenEphys data, preprocess, motion correction, Kilosort4 spike sorting, quality metrics, Allen\u002FIBL curation, AI-assisted visual analysis, for Neuropixels 1.0\u002F2.0 extracellular electrophysiology. Use when working with neural recordings, spike sorting, extracellular electrophysiology, or when the user mentions Neuropixels, SpikeGLX, Open Ephys, Kilosort, quality metrics, or unit curation.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fscientific\u002Fneuropixels-analysis\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fscientific\u002Fneuropixels-analysis\u002FSKILL.md",[],[157,158,159,160,161,162,163,164,41,165,166,167,168,169,170,171,27,172,173,174,122,22],"analysis","neuropixels analysis","spike","metrics","data","open","quality","user","recordings","Neural","motion","visual","quality metrics","unit","sorting","curation","recording","load",{"skillNo":176,"name":177,"nameZh":6,"description":178,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":89,"licenseFileUrl":179,"sourceHtmlUrl":180,"categories":181,"keywords":185,"publishedAt":44},"01M6FDP0PDK6F1GGX395C7625Z","skill-creator","Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, or tool integrations.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fskill-creator\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fskill-creator\u002FSKILL.md",[182],{"categoryNo":183,"categoryName":184,"taxonomyType":96,"taxonomyTypeDesc":97},"01JBSEEDCAT000000000000023","Skill与智能体开发",[186,187,188,189,190,191,192,193,194,195,196,197,198,199,27,122,200,22,201],"skill creator","creator","workflows","create","update","capabilities","knowledge","tool","integrations","specialized","guide","new skill","existing skill","users","Existing","new",{"skillNo":203,"name":204,"nameZh":6,"description":205,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":206,"sourceHtmlUrl":207,"categories":208,"keywords":210,"publishedAt":44},"01M6FDM8Y7VK7NYG9SM7QPA3PW","notebooklm","Use this skill to query your Google NotebookLM notebooks directly from Claude Code for source-grounded, citation-backed answers from Gemini. Browser automation, library management, persistent auth. Drastically reduced hallucinations through document-only responses.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fnotebooklm\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fnotebooklm\u002FSKILL.md",[209],{"categoryNo":183,"categoryName":184,"taxonomyType":96,"taxonomyTypeDesc":97},[204,211,212,213,214,215,216,144,217,218,41,219,220,221,222,122,223],"google","browser automation","gemini","automation","auth","code","library","management","only","document","query","Citation","source",{"skillNo":225,"name":226,"nameZh":6,"description":227,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":89,"licenseFileUrl":228,"sourceHtmlUrl":229,"categories":230,"keywords":237,"publishedAt":44},"01M6FDHPXE10J3XGTCH5HBHS1M","slack-gif-creator","Knowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like \"make me a GIF of X doing Y for Slack.\"","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fenterprise-communication\u002Fslack-gif-creator\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fenterprise-communication\u002Fslack-gif-creator\u002FSKILL.md",[231,234],{"categoryNo":232,"categoryName":233,"taxonomyType":96,"taxonomyTypeDesc":97},"01JBSEEDCAT000000000000005","邮件与沟通",{"categoryNo":235,"categoryName":236,"taxonomyType":96,"taxonomyTypeDesc":97},"01JBSEEDCAT000000000000016","图像与多媒体",[238,239,240,187,241,115,242,243,192,41,244,199,245,246,247,122,248,22],"slack gif creator","slack","gif","validation","constraints","Animation","me","make","Utilities","concepts","request"]