[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6E2W32SA3XY64VFEGFS43TD":3,"skill-same:01M6E2W32SA3XY64VFEGFS43TD":77},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":26,"linkOnly":27,"markdownContent":28,"markdownBody":29,"frontMatter":30,"refName":32,"commitSha":33,"skillDirectory":34,"forksCount":13,"repositoryHtmlUrl":35,"repositoryHomepageUrl":36,"ownerHtmlUrl":37,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":38,"install":39,"resources":52},"01M6E2W32SA3XY64VFEGFS43TD","pytorch-patterns",null,"PyTorch深度学习模式与最佳实践，用于构建稳健、高效且可复现的训练流程、模型架构和数据加载。","7dvq6yrkgs-stack\u002Fplaywright","7dvq6yrkgs-stack","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F251638143?v=4","01M6E061J2AMGWYZ6TK7Y8NM74","01M6E061PQCQJGQMPBERMCFN5G",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fpytorch-patterns\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",[23,24,25],"pytorch","patterns","pytorch patterns","2026-11-10 10:12:30",false,"---\nname: pytorch-patterns\ndescription: PyTorch深度学习模式与最佳实践，用于构建稳健、高效且可复现的训练流程、模型架构和数据加载。\norigin: ECC\n---\n\n# PyTorch 开发模式\n\n构建稳健、高效和可复现深度学习应用的 PyTorch 惯用模式与最佳实践。\n\n## 何时使用\n\n* 编写新的 PyTorch 模型或训练脚本时\n* 评审深度学习代码时\n* 调试训练循环或数据管道时\n* 优化 GPU 内存使用或训练速度时\n* 设置可复现实验时\n\n## 核心原则\n\n### 1. 设备无关代码\n\n始终编写能在 CPU 和 GPU 上运行且不硬编码设备的代码。\n\n```python\n# Good: Device-agnostic\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = MyModel().to(device)\ndata = data.to(device)\n\n# Bad: Hardcoded device\nmodel = MyModel().cuda()  # Crashes if no GPU\ndata = data.cuda()\n```\n\n### 2. 可复现性优先\n\n设置所有随机种子以获得可复现的结果。\n\n```python\n# Good: Full reproducibility setup\ndef set_seed(seed: int = 42) -> None:\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    np.random.seed(seed)\n    random.seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\n\n# Bad: No seed control\nmodel = MyModel()  # Different weights every run\n```\n\n### 3. 显式形状管理\n\n始终记录并验证张量形状。\n\n```python\n# Good: Shape-annotated forward pass\ndef forward(self, x: torch.Tensor) -> torch.Tensor:\n    # x: (batch_size, channels, height, width)\n    x = self.conv1(x)    # -> (batch_size, 32, H, W)\n    x = self.pool(x)     # -> (batch_size, 32, H\u002F\u002F2, W\u002F\u002F2)\n    x = x.view(x.size(0), -1)  # -> (batch_size, 32*H\u002F\u002F2*W\u002F\u002F2)\n    return self.fc(x)    # -> (batch_size, num_classes)\n\n# Bad: No shape tracking\ndef forward(self, x):\n    x = self.conv1(x)\n    x = self.pool(x)\n    x = x.view(x.size(0), -1)  # What size is this?\n    return self.fc(x)           # Will this even work?\n```\n\n## 模型架构模式\n\n### 清晰的 nn.Module 结构\n\n```python\n# Good: Well-organized module\nclass ImageClassifier(nn.Module):\n    def __init__(self, num_classes: int, dropout: float = 0.5) -> None:\n        super().__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(3, 64, kernel_size=3, padding=1),\n            nn.BatchNorm2d(64),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2),\n        )\n        self.classifier = nn.Sequential(\n            nn.Dropout(dropout),\n            nn.Linear(64 * 16 * 16, num_classes),\n        )\n\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        x = self.features(x)\n        x = x.view(x.size(0), -1)\n        return self.classifier(x)\n\n# Bad: Everything in forward\nclass ImageClassifier(nn.Module):\n    def __init__(self):\n        super().__init__()\n\n    def forward(self, x):\n        x = F.conv2d(x, weight=self.make_weight())  # Creates weight each call!\n        return x\n```\n\n### 正确的权重初始化\n\n```python\n# Good: Explicit initialization\ndef _init_weights(self, module: nn.Module) -> None:\n    if isinstance(module, nn.Linear):\n        nn.init.kaiming_normal_(module.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n        if module.bias is not None:\n            nn.init.zeros_(module.bias)\n    elif isinstance(module, nn.Conv2d):\n        nn.init.kaiming_normal_(module.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n    elif isinstance(module, nn.BatchNorm2d):\n        nn.init.ones_(module.weight)\n        nn.init.zeros_(module.bias)\n\nmodel = MyModel()\nmodel.apply(model._init_weights)\n```\n\n## 训练循环模式\n\n### 标准训练循环\n\n```python\n# Good: Complete training loop with best practices\ndef train_one_epoch(\n    model: nn.Module,\n    dataloader: DataLoader,\n    optimizer: torch.optim.Optimizer,\n    criterion: nn.Module,\n    device: torch.device,\n    scaler: torch.amp.GradScaler | None = None,\n) -> float:\n    model.train()  # Always set train mode\n    total_loss = 0.0\n\n    for batch_idx, (data, target) in enumerate(dataloader):\n        data, target = data.to(device), target.to(device)\n\n        optimizer.zero_grad(set_to_none=True)  # More efficient than zero_grad()\n\n        # Mixed precision training\n        with torch.amp.autocast(\"cuda\", enabled=scaler is not None):\n            output = model(data)\n            loss = criterion(output, target)\n\n        if scaler is not None:\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n            scaler.step(optimizer)\n            scaler.update()\n        else:\n            loss.backward()\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n            optimizer.step()\n\n        total_loss += loss.item()\n\n    return total_loss \u002F len(dataloader)\n```\n\n### 验证循环\n\n```python\n# Good: Proper evaluation\n@torch.no_grad()  # More efficient than wrapping in torch.no_grad() block\ndef evaluate(\n    model: nn.Module,\n    dataloader: DataLoader,\n    criterion: nn.Module,\n    device: torch.device,\n) -> tuple[float, float]:\n    model.eval()  # Always set eval mode — disables dropout, uses running BN stats\n    total_loss = 0.0\n    correct = 0\n    total = 0\n\n    for data, target in dataloader:\n        data, target = data.to(device), target.to(device)\n        output = model(data)\n        total_loss += criterion(output, target).item()\n        correct += (output.argmax(1) == target).sum().item()\n        total += target.size(0)\n\n    return total_loss \u002F len(dataloader), correct \u002F total\n```\n\n## 数据管道模式\n\n### 自定义数据集\n\n```python\n# Good: Clean Dataset with type hints\nclass ImageDataset(Dataset):\n    def __init__(\n        self,\n        image_dir: str,\n        labels: dict[str, int],\n        transform: transforms.Compose | None = None,\n    ) -> None:\n        self.image_paths = list(Path(image_dir).glob(\"*.jpg\"))\n        self.labels = labels\n        self.transform = transform\n\n    def __len__(self) -> int:\n        return len(self.image_paths)\n\n    def __getitem__(self, idx: int) -> tuple[torch.Tensor, int]:\n        img = Image.open(self.image_paths[idx]).convert(\"RGB\")\n        label = self.labels[self.image_paths[idx].stem]\n\n        if self.transform:\n            img = self.transform(img)\n\n        return img, label\n```\n\n### 高效的数据加载器配置\n\n```python\n# Good: Optimized DataLoader\ndataloader = DataLoader(\n    dataset,\n    batch_size=32,\n    shuffle=True,            # Shuffle for training\n    num_workers=4,           # Parallel data loading\n    pin_memory=True,         # Faster CPU->GPU transfer\n    persistent_workers=True, # Keep workers alive between epochs\n    drop_last=True,          # Consistent batch sizes for BatchNorm\n)\n\n# Bad: Slow defaults\ndataloader = DataLoader(dataset, batch_size=32)  # num_workers=0, no pin_memory\n```\n\n### 针对变长数据的自定义整理函数\n\n```python\n# Good: Pad sequences in collate_fn\ndef collate_fn(batch: list[tuple[torch.Tensor, int]]) -> tuple[torch.Tensor, torch.Tensor]:\n    sequences, labels = zip(*batch)\n    # Pad to max length in batch\n    padded = nn.utils.rnn.pad_sequence(sequences, batch_first=True, padding_value=0)\n    return padded, torch.tensor(labels)\n\ndataloader = DataLoader(dataset, batch_size=32, collate_fn=collate_fn)\n```\n\n## 检查点模式\n\n### 保存和加载检查点\n\n```python\n# Good: Complete checkpoint with all training state\ndef save_checkpoint(\n    model: nn.Module,\n    optimizer: torch.optim.Optimizer,\n    epoch: int,\n    loss: float,\n    path: str,\n) -> None:\n    torch.save({\n        \"epoch\": epoch,\n        \"model_state_dict\": model.state_dict(),\n        \"optimizer_state_dict\": optimizer.state_dict(),\n        \"loss\": loss,\n    }, path)\n\ndef load_checkpoint(\n    path: str,\n    model: nn.Module,\n    optimizer: torch.optim.Optimizer | None = None,\n) -> dict:\n    checkpoint = torch.load(path, map_location=\"cpu\", weights_only=True)\n    model.load_state_dict(checkpoint[\"model_state_dict\"])\n    if optimizer:\n        optimizer.load_state_dict(checkpoint[\"optimizer_state_dict\"])\n    return checkpoint\n\n# Bad: Only saving model weights (can't resume training)\ntorch.save(model.state_dict(), \"model.pt\")\n```\n\n## 性能优化\n\n### 混合精度训练\n\n```python\n# Good: AMP with GradScaler\nscaler = torch.amp.GradScaler(\"cuda\")\nfor data, target in dataloader:\n    with torch.amp.autocast(\"cuda\"):\n        output = model(data)\n        loss = criterion(output, target)\n    scaler.scale(loss).backward()\n    scaler.step(optimizer)\n    scaler.update()\n    optimizer.zero_grad(set_to_none=True)\n```\n\n### 大模型的梯度检查点\n\n```python\n# Good: Trade compute for memory\nfrom torch.utils.checkpoint import checkpoint\n\nclass LargeModel(nn.Module):\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        # Recompute activations during backward to save memory\n        x = checkpoint(self.block1, x, use_reentrant=False)\n        x = checkpoint(self.block2, x, use_reentrant=False)\n        return self.head(x)\n```\n\n### 使用 torch.compile 加速\n\n```python\n# Good: Compile the model for faster execution (PyTorch 2.0+)\nmodel = MyModel().to(device)\nmodel = torch.compile(model, mode=\"reduce-overhead\")\n\n# Modes: \"default\" (safe), \"reduce-overhead\" (faster), \"max-autotune\" (fastest)\n```\n\n## 快速参考：PyTorch 惯用法\n\n| 惯用法 | 描述 |\n|-------|-------------|\n| `model.train()` \u002F `model.eval()` | 训练\u002F评估前始终设置模式 |\n| `torch.no_grad()` | 推理时禁用梯度 |\n| `optimizer.zero_grad(set_to_none=True)` | 更高效的梯度清零 |\n| `.to(device)` | 设备无关的张量\u002F模型放置 |\n| `torch.amp.autocast` | 混合精度以获得 2 倍速度 |\n| `pin_memory=True` | 更快的 CPU→GPU 数据传输 |\n| `torch.compile` | JIT 编译加速 (2.0+) |\n| `weights_only=True` | 安全的模型加载 |\n| `torch.manual_seed` | 可复现的实验 |\n| `gradient_checkpointing` | 以计算换取内存 |\n\n## 应避免的反模式\n\n```python\n# Bad: Forgetting model.eval() during validation\nmodel.train()\nwith torch.no_grad():\n    output = model(val_data)  # Dropout still active! BatchNorm uses batch stats!\n\n# Good: Always set eval mode\nmodel.eval()\nwith torch.no_grad():\n    output = model(val_data)\n\n# Bad: In-place operations breaking autograd\nx = F.relu(x, inplace=True)  # Can break gradient computation\nx += residual                  # In-place add breaks autograd graph\n\n# Good: Out-of-place operations\nx = F.relu(x)\nx = x + residual\n\n# Bad: Moving data to GPU inside the training loop repeatedly\nfor data, target in dataloader:\n    model = model.cuda()  # Moves model EVERY iteration!\n\n# Good: Move model once before the loop\nmodel = model.to(device)\nfor data, target in dataloader:\n    data, target = data.to(device), target.to(device)\n\n# Bad: Using .item() before backward\nloss = criterion(output, target).item()  # Detaches from graph!\nloss.backward()  # Error: can't backprop through .item()\n\n# Good: Call .item() only for logging\nloss = criterion(output, target)\nloss.backward()\nprint(f\"Loss: {loss.item():.4f}\")  # .item() after backward is fine\n\n# Bad: Not using torch.save properly\ntorch.save(model, \"model.pt\")  # Saves entire model (fragile, not portable)\n\n# Good: Save state_dict\ntorch.save(model.state_dict(), \"model.pt\")\n```\n\n**请记住**：PyTorch 代码应做到设备无关、可复现且内存意识强。如有疑问，请使用 `torch.profiler` 进行分析，并使用 `torch.cuda.memory_summary()` 检查 GPU 内存。\n","# PyTorch 开发模式\n\n构建稳健、高效和可复现深度学习应用的 PyTorch 惯用模式与最佳实践。\n\n## 何时使用\n\n* 编写新的 PyTorch 模型或训练脚本时\n* 评审深度学习代码时\n* 调试训练循环或数据管道时\n* 优化 GPU 内存使用或训练速度时\n* 设置可复现实验时\n\n## 核心原则\n\n### 1. 设备无关代码\n\n始终编写能在 CPU 和 GPU 上运行且不硬编码设备的代码。\n\n```python\n# Good: Device-agnostic\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = MyModel().to(device)\ndata = data.to(device)\n\n# Bad: Hardcoded device\nmodel = MyModel().cuda()  # Crashes if no GPU\ndata = data.cuda()\n```\n\n### 2. 可复现性优先\n\n设置所有随机种子以获得可复现的结果。\n\n```python\n# Good: Full reproducibility setup\ndef set_seed(seed: int = 42) -> None:\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    np.random.seed(seed)\n    random.seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\n\n# Bad: No seed control\nmodel = MyModel()  # Different weights every run\n```\n\n### 3. 显式形状管理\n\n始终记录并验证张量形状。\n\n```python\n# Good: Shape-annotated forward pass\ndef forward(self, x: torch.Tensor) -> torch.Tensor:\n    # x: (batch_size, channels, height, width)\n    x = self.conv1(x)    # -> (batch_size, 32, H, W)\n    x = self.pool(x)     # -> (batch_size, 32, H\u002F\u002F2, W\u002F\u002F2)\n    x = x.view(x.size(0), -1)  # -> (batch_size, 32*H\u002F\u002F2*W\u002F\u002F2)\n    return self.fc(x)    # -> (batch_size, num_classes)\n\n# Bad: No shape tracking\ndef forward(self, x):\n    x = self.conv1(x)\n    x = self.pool(x)\n    x = x.view(x.size(0), -1)  # What size is this?\n    return self.fc(x)           # Will this even work?\n```\n\n## 模型架构模式\n\n### 清晰的 nn.Module 结构\n\n```python\n# Good: Well-organized module\nclass ImageClassifier(nn.Module):\n    def __init__(self, num_classes: int, dropout: float = 0.5) -> None:\n        super().__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(3, 64, kernel_size=3, padding=1),\n            nn.BatchNorm2d(64),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2),\n        )\n        self.classifier = nn.Sequential(\n            nn.Dropout(dropout),\n            nn.Linear(64 * 16 * 16, num_classes),\n        )\n\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        x = self.features(x)\n        x = x.view(x.size(0), -1)\n        return self.classifier(x)\n\n# Bad: Everything in forward\nclass ImageClassifier(nn.Module):\n    def __init__(self):\n        super().__init__()\n\n    def forward(self, x):\n        x = F.conv2d(x, weight=self.make_weight())  # Creates weight each call!\n        return x\n```\n\n### 正确的权重初始化\n\n```python\n# Good: Explicit initialization\ndef _init_weights(self, module: nn.Module) -> None:\n    if isinstance(module, nn.Linear):\n        nn.init.kaiming_normal_(module.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n        if module.bias is not None:\n            nn.init.zeros_(module.bias)\n    elif isinstance(module, nn.Conv2d):\n        nn.init.kaiming_normal_(module.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n    elif isinstance(module, nn.BatchNorm2d):\n        nn.init.ones_(module.weight)\n        nn.init.zeros_(module.bias)\n\nmodel = MyModel()\nmodel.apply(model._init_weights)\n```\n\n## 训练循环模式\n\n### 标准训练循环\n\n```python\n# Good: Complete training loop with best practices\ndef train_one_epoch(\n    model: nn.Module,\n    dataloader: DataLoader,\n    optimizer: torch.optim.Optimizer,\n    criterion: nn.Module,\n    device: torch.device,\n    scaler: torch.amp.GradScaler | None = None,\n) -> float:\n    model.train()  # Always set train mode\n    total_loss = 0.0\n\n    for batch_idx, (data, target) in enumerate(dataloader):\n        data, target = data.to(device), target.to(device)\n\n        optimizer.zero_grad(set_to_none=True)  # More efficient than zero_grad()\n\n        # Mixed precision training\n        with torch.amp.autocast(\"cuda\", enabled=scaler is not None):\n            output = model(data)\n            loss = criterion(output, target)\n\n        if scaler is not None:\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n            scaler.step(optimizer)\n            scaler.update()\n        else:\n            loss.backward()\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n            optimizer.step()\n\n        total_loss += loss.item()\n\n    return total_loss \u002F len(dataloader)\n```\n\n### 验证循环\n\n```python\n# Good: Proper evaluation\n@torch.no_grad()  # More efficient than wrapping in torch.no_grad() block\ndef evaluate(\n    model: nn.Module,\n    dataloader: DataLoader,\n    criterion: nn.Module,\n    device: torch.device,\n) -> tuple[float, float]:\n    model.eval()  # Always set eval mode — disables dropout, uses running BN stats\n    total_loss = 0.0\n    correct = 0\n    total = 0\n\n    for data, target in dataloader:\n        data, target = data.to(device), target.to(device)\n        output = model(data)\n        total_loss += criterion(output, target).item()\n        correct += (output.argmax(1) == target).sum().item()\n        total += target.size(0)\n\n    return total_loss \u002F len(dataloader), correct \u002F total\n```\n\n## 数据管道模式\n\n### 自定义数据集\n\n```python\n# Good: Clean Dataset with type hints\nclass ImageDataset(Dataset):\n    def __init__(\n        self,\n        image_dir: str,\n        labels: dict[str, int],\n        transform: transforms.Compose | None = None,\n    ) -> None:\n        self.image_paths = list(Path(image_dir).glob(\"*.jpg\"))\n        self.labels = labels\n        self.transform = transform\n\n    def __len__(self) -> int:\n        return len(self.image_paths)\n\n    def __getitem__(self, idx: int) -> tuple[torch.Tensor, int]:\n        img = Image.open(self.image_paths[idx]).convert(\"RGB\")\n        label = self.labels[self.image_paths[idx].stem]\n\n        if self.transform:\n            img = self.transform(img)\n\n        return img, label\n```\n\n### 高效的数据加载器配置\n\n```python\n# Good: Optimized DataLoader\ndataloader = DataLoader(\n    dataset,\n    batch_size=32,\n    shuffle=True,            # Shuffle for training\n    num_workers=4,           # Parallel data loading\n    pin_memory=True,         # Faster CPU->GPU transfer\n    persistent_workers=True, # Keep workers alive between epochs\n    drop_last=True,          # Consistent batch sizes for BatchNorm\n)\n\n# Bad: Slow defaults\ndataloader = DataLoader(dataset, batch_size=32)  # num_workers=0, no pin_memory\n```\n\n### 针对变长数据的自定义整理函数\n\n```python\n# Good: Pad sequences in collate_fn\ndef collate_fn(batch: list[tuple[torch.Tensor, int]]) -> tuple[torch.Tensor, torch.Tensor]:\n    sequences, labels = zip(*batch)\n    # Pad to max length in batch\n    padded = nn.utils.rnn.pad_sequence(sequences, batch_first=True, padding_value=0)\n    return padded, torch.tensor(labels)\n\ndataloader = DataLoader(dataset, batch_size=32, collate_fn=collate_fn)\n```\n\n## 检查点模式\n\n### 保存和加载检查点\n\n```python\n# Good: Complete checkpoint with all training state\ndef save_checkpoint(\n    model: nn.Module,\n    optimizer: torch.optim.Optimizer,\n    epoch: int,\n    loss: float,\n    path: str,\n) -> None:\n    torch.save({\n        \"epoch\": epoch,\n        \"model_state_dict\": model.state_dict(),\n        \"optimizer_state_dict\": optimizer.state_dict(),\n        \"loss\": loss,\n    }, path)\n\ndef load_checkpoint(\n    path: str,\n    model: nn.Module,\n    optimizer: torch.optim.Optimizer | None = None,\n) -> dict:\n    checkpoint = torch.load(path, map_location=\"cpu\", weights_only=True)\n    model.load_state_dict(checkpoint[\"model_state_dict\"])\n    if optimizer:\n        optimizer.load_state_dict(checkpoint[\"optimizer_state_dict\"])\n    return checkpoint\n\n# Bad: Only saving model weights (can't resume training)\ntorch.save(model.state_dict(), \"model.pt\")\n```\n\n## 性能优化\n\n### 混合精度训练\n\n```python\n# Good: AMP with GradScaler\nscaler = torch.amp.GradScaler(\"cuda\")\nfor data, target in dataloader:\n    with torch.amp.autocast(\"cuda\"):\n        output = model(data)\n        loss = criterion(output, target)\n    scaler.scale(loss).backward()\n    scaler.step(optimizer)\n    scaler.update()\n    optimizer.zero_grad(set_to_none=True)\n```\n\n### 大模型的梯度检查点\n\n```python\n# Good: Trade compute for memory\nfrom torch.utils.checkpoint import checkpoint\n\nclass LargeModel(nn.Module):\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        # Recompute activations during backward to save memory\n        x = checkpoint(self.block1, x, use_reentrant=False)\n        x = checkpoint(self.block2, x, use_reentrant=False)\n        return self.head(x)\n```\n\n### 使用 torch.compile 加速\n\n```python\n# Good: Compile the model for faster execution (PyTorch 2.0+)\nmodel = MyModel().to(device)\nmodel = torch.compile(model, mode=\"reduce-overhead\")\n\n# Modes: \"default\" (safe), \"reduce-overhead\" (faster), \"max-autotune\" (fastest)\n```\n\n## 快速参考：PyTorch 惯用法\n\n| 惯用法 | 描述 |\n|-------|-------------|\n| `model.train()` \u002F `model.eval()` | 训练\u002F评估前始终设置模式 |\n| `torch.no_grad()` | 推理时禁用梯度 |\n| `optimizer.zero_grad(set_to_none=True)` | 更高效的梯度清零 |\n| `.to(device)` | 设备无关的张量\u002F模型放置 |\n| `torch.amp.autocast` | 混合精度以获得 2 倍速度 |\n| `pin_memory=True` | 更快的 CPU→GPU 数据传输 |\n| `torch.compile` | JIT 编译加速 (2.0+) |\n| `weights_only=True` | 安全的模型加载 |\n| `torch.manual_seed` | 可复现的实验 |\n| `gradient_checkpointing` | 以计算换取内存 |\n\n## 应避免的反模式\n\n```python\n# Bad: Forgetting model.eval() during validation\nmodel.train()\nwith torch.no_grad():\n    output = model(val_data)  # Dropout still active! BatchNorm uses batch stats!\n\n# Good: Always set eval mode\nmodel.eval()\nwith torch.no_grad():\n    output = model(val_data)\n\n# Bad: In-place operations breaking autograd\nx = F.relu(x, inplace=True)  # Can break gradient computation\nx += residual                  # In-place add breaks autograd graph\n\n# Good: Out-of-place operations\nx = F.relu(x)\nx = x + residual\n\n# Bad: Moving data to GPU inside the training loop repeatedly\nfor data, target in dataloader:\n    model = model.cuda()  # Moves model EVERY iteration!\n\n# Good: Move model once before the loop\nmodel = model.to(device)\nfor data, target in dataloader:\n    data, target = data.to(device), target.to(device)\n\n# Bad: Using .item() before backward\nloss = criterion(output, target).item()  # Detaches from graph!\nloss.backward()  # Error: can't backprop through .item()\n\n# Good: Call .item() only for logging\nloss = criterion(output, target)\nloss.backward()\nprint(f\"Loss: {loss.item():.4f}\")  # .item() after backward is fine\n\n# Bad: Not using torch.save properly\ntorch.save(model, \"model.pt\")  # Saves entire model (fragile, not portable)\n\n# Good: Save state_dict\ntorch.save(model.state_dict(), \"model.pt\")\n```\n\n**请记住**：PyTorch 代码应做到设备无关、可复现且内存意识强。如有疑问，请使用 `torch.profiler` 进行分析，并使用 `torch.cuda.memory_summary()` 检查 GPU 内存。\n",{"name":5,"origin":31,"description":7},"ECC","main","a2e3284ed4488780397ee2f78217366e11470dfb","skills\u002Fpytorch-patterns","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright","https:\u002F\u002Fplaywright.dev","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack","2026-04-25 06:04:45",{"source":8,"skillName":5,"commands":40,"prompt":50,"zipAvailable":51},[41,44,47],{"tool":42,"command":43},"npx","npx skills add 7dvq6yrkgs-stack\u002Fplaywright --skill pytorch-patterns",{"tool":45,"command":46},"bunx","bunx skills add 7dvq6yrkgs-stack\u002Fplaywright --skill pytorch-patterns",{"tool":48,"command":49},"pnpm","pnpm dlx skills add 7dvq6yrkgs-stack\u002Fplaywright --skill pytorch-patterns","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fpytorch-patterns\u002FSKILL.md",true,[53,59,63,67,73],{"path":54,"resourceType":55,"resourceTypeDesc":56,"sizeBytes":57,"sourceHtmlUrl":58},".skillfish.json","REFERENCE","参考文档",250,"https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fpytorch-patterns\u002F.skillfish.json",{"path":60,"resourceType":55,"resourceTypeDesc":56,"sizeBytes":61,"sourceHtmlUrl":62},"references\u002F21cfr11-compliance-guide.md",13369,"https:\u002F\u002Fgithub.com\u002Fagency-black\u002FSkills\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fskills\u002Fquality-documentation-manager\u002Freferences\u002F21cfr11-compliance-guide.md",{"path":64,"resourceType":55,"resourceTypeDesc":56,"sizeBytes":65,"sourceHtmlUrl":66},"references\u002Fdocument-control-procedures.md",11984,"https:\u002F\u002Fgithub.com\u002Fagency-black\u002FSkills\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fskills\u002Fquality-documentation-manager\u002Freferences\u002Fdocument-control-procedures.md",{"path":68,"resourceType":69,"resourceTypeDesc":70,"sizeBytes":71,"sourceHtmlUrl":72},"scripts\u002Fdocument_validator.py","SCRIPT","脚本",22838,"https:\u002F\u002Fgithub.com\u002Fagency-black\u002FSkills\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fskills\u002Fquality-documentation-manager\u002Fscripts\u002Fdocument_validator.py",{"path":74,"resourceType":69,"resourceTypeDesc":70,"sizeBytes":75,"sourceHtmlUrl":76},"scripts\u002Fdocument_version_control.py",17024,"https:\u002F\u002Fgithub.com\u002Fagency-black\u002FSkills\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fskills\u002Fquality-documentation-manager\u002Fscripts\u002Fdocument_version_control.py",[78,108,118,128,143,158],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":82,"categories":83,"keywords":96,"publishedAt":26},"01M6E3XK4PRPMQHM7Y98FQPTY1","zyte-api-automation","Automate Zyte API tasks via Rube MCP (Composio). Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fzyte-api-automation\u002FSKILL.md",[84,85,88,91],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":86,"categoryName":87,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":89,"categoryName":90,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":92,"categoryName":93,"taxonomyType":94,"taxonomyTypeDesc":95},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[97,98,99,100,101,102,103,104,105,106,107],"zyte api automation","api","automation","MCP","tasks","search","Tools","first","automate","composio","via",{"skillNo":109,"name":110,"nameZh":6,"description":111,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":112,"categories":113,"keywords":116,"publishedAt":26},"01M6E3XFX6QSTR1QR59NFXJTMA","zylvie-automation","Automate Zylvie tasks via Rube MCP (Composio). Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fzylvie-automation\u002FSKILL.md",[114,115],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":89,"categoryName":90,"taxonomyType":20,"taxonomyTypeDesc":21},[117,99,105,100,101,102,103,104,106,107],"zylvie automation",{"skillNo":119,"name":120,"nameZh":6,"description":121,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":122,"categories":123,"keywords":126,"publishedAt":26},"01M6E3XCX2GDYYCHN58ZJHT37M","zoominfo-automation","Automate Zoominfo tasks via Rube MCP (Composio). Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fzoominfo-automation\u002FSKILL.md",[124,125],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":89,"categoryName":90,"taxonomyType":20,"taxonomyTypeDesc":21},[127,99,102,103,104,105,106,107,101,100],"zoominfo automation",{"skillNo":129,"name":130,"nameZh":6,"description":131,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":132,"categories":133,"keywords":136,"publishedAt":26},"01M6E3X9WJ2T73216WQHHTB9BD","zoho_mail-automation","Automate Zoho Mail tasks via Rube MCP (Composio): email sending, folders, labels, and mailbox management. Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fzoho_mail-automation\u002FSKILL.md",[134,135],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":89,"categoryName":90,"taxonomyType":20,"taxonomyTypeDesc":21},[137,138,139,99,100,101,102,140,103,104,105,141,106,142,107],"mail","zoho mail automation","zoho","email","management","mailbox",{"skillNo":144,"name":145,"nameZh":6,"description":146,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":147,"categories":148,"keywords":154,"publishedAt":26},"01M6E3X719W648XZBW1EC6JQ64","zoho_invoice-automation","Automate Zoho Invoice tasks via Rube MCP (Composio): invoices, estimates, expenses, clients, and payment tracking. Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fzoho_invoice-automation\u002FSKILL.md",[149,150,151],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":89,"categoryName":90,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":152,"categoryName":153,"taxonomyType":94,"taxonomyTypeDesc":95},"01JBSEEDCAT000000000000040","财务",[155,139,99,101,102,103,104,105,156,157,106,107,100],"zoho invoice automation","tracking","payment",{"skillNo":159,"name":160,"nameZh":6,"description":161,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":162,"categories":163,"keywords":166,"publishedAt":26},"01M6E3X28JYTC3QTK32AQF7BYB","zoho_inventory-automation","Automate Zoho Inventory tasks via Rube MCP (Composio): items, orders, warehouses, shipments, and stock management. Always search tools first for current schemas.","https:\u002F\u002Fgithub.com\u002F7dvq6yrkgs-stack\u002Fplaywright\u002Fblob\u002Fmain\u002Fskills\u002Fzoho_inventory-automation\u002FSKILL.md",[164,165],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":89,"categoryName":90,"taxonomyType":20,"taxonomyTypeDesc":21},[99,167,139,168,104,105,103,102,101,100,106,107,169,170,141],"inventory","zoho inventory automation","stock","orders"]