[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6DCQ0SWBJ1CSRRFC6Y3JZKV":3,"skill-same:01M6DCQ0SWBJ1CSRRFC6Y3JZKV":80},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":58,"commitSha":59,"skillDirectory":60,"forksCount":61,"repositoryHtmlUrl":62,"repositoryHomepageUrl":6,"ownerHtmlUrl":63,"ownerBlogUrl":6,"ownerDescription":64,"updatedAt":65,"install":66,"resources":79},"01M6DCQ0SWBJ1CSRRFC6Y3JZKV","segment-anything-model",null,"SAM: zero-shot image segmentation via points, boxes, masks.","zapabob\u002FSkills","zapabob","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F130830376?v=4","01M6DBMYZAGRTFQDSQYW3W764P","峯岸 亮","01M6DBMZ3GKDSJ01NZ9XTZD7JD",2,"MIT","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Farchive\u002Fskill-md-reviewed\u002Fsha256\u002F19\u002F198715c51b662f92e21fcdd4f8762427e9277827a0dad0fcfe3314264a3f845d\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"model","segment anything model","image","macos","linux","research","windows","multimodal","ZeRO","Computer Vision","Transformers","segmentation","computer","vision","boxes","2026-11-10 10:12:30",false,"---\r\nname: segment-anything-model\r\ndescription: \"SAM: zero-shot image segmentation via points, boxes, masks.\"\r\nversion: 1.0.0\r\nauthor: Orchestra Research\r\nlicense: MIT\r\ndependencies: [segment-anything, transformers>=4.30.0, torch>=1.7.0]\r\nplatforms: [linux, macos, windows]\r\nmetadata:\r\n  hermes:\r\n    tags: [Multimodal, Image Segmentation, Computer Vision, SAM, Zero-Shot]\r\n\r\n---\r\n\r\n# Segment Anything Model (SAM)\r\n\r\nComprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.\r\n\r\n## When to use SAM\r\n\r\n**Use SAM when:**\r\n- Need to segment any object in images without task-specific training\r\n- Building interactive annotation tools with point\u002Fbox prompts\r\n- Generating training data for other vision models\r\n- Need zero-shot transfer to new image domains\r\n- Building object detection\u002Fsegmentation pipelines\r\n- Processing medical, satellite, or domain-specific images\r\n\r\n**Key features:**\r\n- **Zero-shot segmentation**: Works on any image domain without fine-tuning\r\n- **Flexible prompts**: Points, bounding boxes, or previous masks\r\n- **Automatic segmentation**: Generate all object masks automatically\r\n- **High quality**: Trained on 1.1 billion masks from 11 million images\r\n- **Multiple model sizes**: ViT-B (fastest), ViT-L, ViT-H (most accurate)\r\n- **ONNX export**: Deploy in browsers and edge devices\r\n\r\n**Use alternatives instead:**\r\n- **YOLO\u002FDetectron2**: For real-time object detection with classes\r\n- **Mask2Former**: For semantic\u002Fpanoptic segmentation with categories\r\n- **GroundingDINO + SAM**: For text-prompted segmentation\r\n- **SAM 2**: For video segmentation tasks\r\n\r\n## Quick start\r\n\r\n### Installation\r\n\r\n```bash\r\n# From GitHub\r\npip install git+https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything.git\r\n\r\n# Optional dependencies\r\npip install opencv-python pycocotools matplotlib\r\n\r\n# Or use HuggingFace transformers\r\npip install transformers\r\n```\r\n\r\n### Download checkpoints\r\n\r\n```bash\r\n# ViT-H (largest, most accurate) - 2.4GB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_h_4b8939.pth\r\n\r\n# ViT-L (medium) - 1.2GB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_l_0b3195.pth\r\n\r\n# ViT-B (smallest, fastest) - 375MB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_b_01ec64.pth\r\n```\r\n\r\n### Basic usage with SamPredictor\r\n\r\n```python\r\nimport numpy as np\r\nfrom segment_anything import sam_model_registry, SamPredictor\r\n\r\n# Load model\r\nsam = sam_model_registry[\"vit_h\"](checkpoint=\"sam_vit_h_4b8939.pth\")\r\nsam.to(device=\"cuda\")\r\n\r\n# Create predictor\r\npredictor = SamPredictor(sam)\r\n\r\n# Set image (computes embeddings once)\r\nimage = cv2.imread(\"image.jpg\")\r\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\r\npredictor.set_image(image)\r\n\r\n# Predict with point prompts\r\ninput_point = np.array([[500, 375]])  # (x, y) coordinates\r\ninput_label = np.array([1])  # 1 = foreground, 0 = background\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_point,\r\n    point_labels=input_label,\r\n    multimask_output=True  # Returns 3 mask options\r\n)\r\n\r\n# Select best mask\r\nbest_mask = masks[np.argmax(scores)]\r\n```\r\n\r\n### HuggingFace Transformers\r\n\r\n```python\r\nimport torch\r\nfrom PIL import Image\r\nfrom transformers import SamModel, SamProcessor\r\n\r\n# Load model and processor\r\nmodel = SamModel.from_pretrained(\"facebook\u002Fsam-vit-huge\")\r\nprocessor = SamProcessor.from_pretrained(\"facebook\u002Fsam-vit-huge\")\r\nmodel.to(\"cuda\")\r\n\r\n# Process image with point prompt\r\nimage = Image.open(\"image.jpg\")\r\ninput_points = [[[450, 600]]]  # Batch of points\r\n\r\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\")\r\ninputs = {k: v.to(\"cuda\") for k, v in inputs.items()}\r\n\r\n# Generate masks\r\nwith torch.no_grad():\r\n    outputs = model(**inputs)\r\n\r\n# Post-process masks to original size\r\nmasks = processor.image_processor.post_process_masks(\r\n    outputs.pred_masks.cpu(),\r\n    inputs[\"original_sizes\"].cpu(),\r\n    inputs[\"reshaped_input_sizes\"].cpu()\r\n)\r\n```\r\n\r\n## Core concepts\r\n\r\n### Model architecture\r\n\r\n\u003C!-- ascii-guard-ignore -->\r\n```\r\nSAM Architecture:\r\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\r\n│  Image Encoder  │────▶│ Prompt Encoder  │────▶│  Mask Decoder   │\r\n│     (ViT)       │     │ (Points\u002FBoxes)  │     │ (Transformer)   │\r\n└─────────────────┘     └─────────────────┘     └─────────────────┘\r\n        │                       │                       │\r\n   Image Embeddings      Prompt Embeddings         Masks + IoU\r\n   (computed once)       (per prompt)             predictions\r\n```\r\n\u003C!-- ascii-guard-ignore-end -->\r\n\r\n### Model variants\r\n\r\n| Model | Checkpoint | Size | Speed | Accuracy |\r\n|-------|------------|------|-------|----------|\r\n| ViT-H | `vit_h` | 2.4 GB | Slowest | Best |\r\n| ViT-L | `vit_l` | 1.2 GB | Medium | Good |\r\n| ViT-B | `vit_b` | 375 MB | Fastest | Good |\r\n\r\n### Prompt types\r\n\r\n| Prompt | Description | Use Case |\r\n|--------|-------------|----------|\r\n| Point (foreground) | Click on object | Single object selection |\r\n| Point (background) | Click outside object | Exclude regions |\r\n| Bounding box | Rectangle around object | Larger objects |\r\n| Previous mask | Low-res mask input | Iterative refinement |\r\n\r\n## Interactive segmentation\r\n\r\n### Point prompts\r\n\r\n```python\r\n# Single foreground point\r\ninput_point = np.array([[500, 375]])\r\ninput_label = np.array([1])\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_point,\r\n    point_labels=input_label,\r\n    multimask_output=True\r\n)\r\n\r\n# Multiple points (foreground + background)\r\ninput_points = np.array([[500, 375], [600, 400], [450, 300]])\r\ninput_labels = np.array([1, 1, 0])  # 2 foreground, 1 background\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_points,\r\n    point_labels=input_labels,\r\n    multimask_output=False  # Single mask when prompts are clear\r\n)\r\n```\r\n\r\n### Box prompts\r\n\r\n```python\r\n# Bounding box [x1, y1, x2, y2]\r\ninput_box = np.array([425, 600, 700, 875])\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    box=input_box,\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n### Combined prompts\r\n\r\n```python\r\n# Box + points for precise control\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375]]),\r\n    point_labels=np.array([1]),\r\n    box=np.array([400, 300, 700, 600]),\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n### Iterative refinement\r\n\r\n```python\r\n# Initial prediction\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375]]),\r\n    point_labels=np.array([1]),\r\n    multimask_output=True\r\n)\r\n\r\n# Refine with additional point using previous mask\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375], [550, 400]]),\r\n    point_labels=np.array([1, 0]),  # Add background point\r\n    mask_input=logits[np.argmax(scores)][None, :, :],  # Use best mask\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n## Automatic mask generation\r\n\r\n### Basic automatic segmentation\r\n\r\n```python\r\nfrom segment_anything import SamAutomaticMaskGenerator\r\n\r\n# Create generator\r\nmask_generator = SamAutomaticMaskGenerator(sam)\r\n\r\n# Generate all masks\r\nmasks = mask_generator.generate(image)\r\n\r\n# Each mask contains:\r\n# - segmentation: binary mask\r\n# - bbox: [x, y, w, h]\r\n# - area: pixel count\r\n# - predicted_iou: quality score\r\n# - stability_score: robustness score\r\n# - point_coords: generating point\r\n```\r\n\r\n### Customized generation\r\n\r\n```python\r\nmask_generator = SamAutomaticMaskGenerator(\r\n    model=sam,\r\n    points_per_side=32,          # Grid density (more = more masks)\r\n    pred_iou_thresh=0.88,        # Quality threshold\r\n    stability_score_thresh=0.95,  # Stability threshold\r\n    crop_n_layers=1,             # Multi-scale crops\r\n    crop_n_points_downscale_factor=2,\r\n    min_mask_region_area=100,    # Remove tiny masks\r\n)\r\n\r\nmasks = mask_generator.generate(image)\r\n```\r\n\r\n### Filtering masks\r\n\r\n```python\r\n# Sort by area (largest first)\r\nmasks = sorted(masks, key=lambda x: x['area'], reverse=True)\r\n\r\n# Filter by predicted IoU\r\nhigh_quality = [m for m in masks if m['predicted_iou'] > 0.9]\r\n\r\n# Filter by stability score\r\nstable_masks = [m for m in masks if m['stability_score'] > 0.95]\r\n```\r\n\r\n## Batched inference\r\n\r\n### Multiple images\r\n\r\n```python\r\n# Process multiple images efficiently\r\nimages = [cv2.imread(f\"image_{i}.jpg\") for i in range(10)]\r\n\r\nall_masks = []\r\nfor image in images:\r\n    predictor.set_image(image)\r\n    masks, _, _ = predictor.predict(\r\n        point_coords=np.array([[500, 375]]),\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n    all_masks.append(masks)\r\n```\r\n\r\n### Multiple prompts per image\r\n\r\n```python\r\n# Process multiple prompts efficiently (one image encoding)\r\npredictor.set_image(image)\r\n\r\n# Batch of point prompts\r\npoints = [\r\n    np.array([[100, 100]]),\r\n    np.array([[200, 200]]),\r\n    np.array([[300, 300]])\r\n]\r\n\r\nall_masks = []\r\nfor point in points:\r\n    masks, scores, _ = predictor.predict(\r\n        point_coords=point,\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n    all_masks.append(masks[np.argmax(scores)])\r\n```\r\n\r\n## ONNX deployment\r\n\r\n### Export model\r\n\r\n```bash\r\npython scripts\u002Fexport_onnx_model.py \\\r\n    --checkpoint sam_vit_h_4b8939.pth \\\r\n    --model-type vit_h \\\r\n    --output sam_onnx.onnx \\\r\n    --return-single-mask\r\n```\r\n\r\n### Use ONNX model\r\n\r\n```python\r\nimport onnxruntime\r\n\r\n# Load ONNX model\r\nort_session = onnxruntime.InferenceSession(\"sam_onnx.onnx\")\r\n\r\n# Run inference (image embeddings computed separately)\r\nmasks = ort_session.run(\r\n    None,\r\n    {\r\n        \"image_embeddings\": image_embeddings,\r\n        \"point_coords\": point_coords,\r\n        \"point_labels\": point_labels,\r\n        \"mask_input\": np.zeros((1, 1, 256, 256), dtype=np.float32),\r\n        \"has_mask_input\": np.array([0], dtype=np.float32),\r\n        \"orig_im_size\": np.array([h, w], dtype=np.float32)\r\n    }\r\n)\r\n```\r\n\r\n## Common workflows\r\n\r\n### Workflow 1: Annotation tool\r\n\r\n```python\r\nimport cv2\r\n\r\n# Load model\r\npredictor = SamPredictor(sam)\r\npredictor.set_image(image)\r\n\r\ndef on_click(event, x, y, flags, param):\r\n    if event == cv2.EVENT_LBUTTONDOWN:\r\n        # Foreground point\r\n        masks, scores, _ = predictor.predict(\r\n            point_coords=np.array([[x, y]]),\r\n            point_labels=np.array([1]),\r\n            multimask_output=True\r\n        )\r\n        # Display best mask\r\n        display_mask(masks[np.argmax(scores)])\r\n```\r\n\r\n### Workflow 2: Object extraction\r\n\r\n```python\r\ndef extract_object(image, point):\r\n    \"\"\"Extract object at point with transparent background.\"\"\"\r\n    predictor.set_image(image)\r\n\r\n    masks, scores, _ = predictor.predict(\r\n        point_coords=np.array([point]),\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n\r\n    best_mask = masks[np.argmax(scores)]\r\n\r\n    # Create RGBA output\r\n    rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)\r\n    rgba[:, :, :3] = image\r\n    rgba[:, :, 3] = best_mask * 255\r\n\r\n    return rgba\r\n```\r\n\r\n### Workflow 3: Medical image segmentation\r\n\r\n```python\r\n# Process medical images (grayscale to RGB)\r\nmedical_image = cv2.imread(\"scan.png\", cv2.IMREAD_GRAYSCALE)\r\nrgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)\r\n\r\npredictor.set_image(rgb_image)\r\n\r\n# Segment region of interest\r\nmasks, scores, _ = predictor.predict(\r\n    box=np.array([x1, y1, x2, y2]),  # ROI bounding box\r\n    multimask_output=True\r\n)\r\n```\r\n\r\n## Output format\r\n\r\n### Mask data structure\r\n\r\n```python\r\n# SamAutomaticMaskGenerator output\r\n{\r\n    \"segmentation\": np.ndarray,  # H×W binary mask\r\n    \"bbox\": [x, y, w, h],        # Bounding box\r\n    \"area\": int,                 # Pixel count\r\n    \"predicted_iou\": float,      # 0-1 quality score\r\n    \"stability_score\": float,    # 0-1 robustness score\r\n    \"crop_box\": [x, y, w, h],    # Generation crop region\r\n    \"point_coords\": [[x, y]],    # Input point\r\n}\r\n```\r\n\r\n### COCO RLE format\r\n\r\n```python\r\nfrom pycocotools import mask as mask_utils\r\n\r\n# Encode mask to RLE\r\nrle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))\r\nrle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\r\n\r\n# Decode RLE to mask\r\ndecoded_mask = mask_utils.decode(rle)\r\n```\r\n\r\n## Performance optimization\r\n\r\n### GPU memory\r\n\r\n```python\r\n# Use smaller model for limited VRAM\r\nsam = sam_model_registry[\"vit_b\"](checkpoint=\"sam_vit_b_01ec64.pth\")\r\n\r\n# Process images in batches\r\n# Clear CUDA cache between large batches\r\ntorch.cuda.empty_cache()\r\n```\r\n\r\n### Speed optimization\r\n\r\n```python\r\n# Use half precision\r\nsam = sam.half()\r\n\r\n# Reduce points for automatic generation\r\nmask_generator = SamAutomaticMaskGenerator(\r\n    model=sam,\r\n    points_per_side=16,  # Default is 32\r\n)\r\n\r\n# Use ONNX for deployment\r\n# Export with --return-single-mask for faster inference\r\n```\r\n\r\n## Common issues\r\n\r\n| Issue | Solution |\r\n|-------|----------|\r\n| Out of memory | Use ViT-B model, reduce image size |\r\n| Slow inference | Use ViT-B, reduce points_per_side |\r\n| Poor mask quality | Try different prompts, use box + points |\r\n| Edge artifacts | Use stability_score filtering |\r\n| Small objects missed | Increase points_per_side |\r\n\r\n## References\r\n\r\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Batching, fine-tuning, integration\r\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\r\n\r\n## Resources\r\n\r\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything\r\n- **Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.02643\r\n- **Demo**: https:\u002F\u002Fsegment-anything.com\r\n- **SAM 2 (Video)**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2\r\n- **HuggingFace**: https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fsam-vit-huge\r\n","# Segment Anything Model (SAM)\r\n\r\nComprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.\r\n\r\n## When to use SAM\r\n\r\n**Use SAM when:**\r\n- Need to segment any object in images without task-specific training\r\n- Building interactive annotation tools with point\u002Fbox prompts\r\n- Generating training data for other vision models\r\n- Need zero-shot transfer to new image domains\r\n- Building object detection\u002Fsegmentation pipelines\r\n- Processing medical, satellite, or domain-specific images\r\n\r\n**Key features:**\r\n- **Zero-shot segmentation**: Works on any image domain without fine-tuning\r\n- **Flexible prompts**: Points, bounding boxes, or previous masks\r\n- **Automatic segmentation**: Generate all object masks automatically\r\n- **High quality**: Trained on 1.1 billion masks from 11 million images\r\n- **Multiple model sizes**: ViT-B (fastest), ViT-L, ViT-H (most accurate)\r\n- **ONNX export**: Deploy in browsers and edge devices\r\n\r\n**Use alternatives instead:**\r\n- **YOLO\u002FDetectron2**: For real-time object detection with classes\r\n- **Mask2Former**: For semantic\u002Fpanoptic segmentation with categories\r\n- **GroundingDINO + SAM**: For text-prompted segmentation\r\n- **SAM 2**: For video segmentation tasks\r\n\r\n## Quick start\r\n\r\n### Installation\r\n\r\n```bash\r\n# From GitHub\r\npip install git+https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything.git\r\n\r\n# Optional dependencies\r\npip install opencv-python pycocotools matplotlib\r\n\r\n# Or use HuggingFace transformers\r\npip install transformers\r\n```\r\n\r\n### Download checkpoints\r\n\r\n```bash\r\n# ViT-H (largest, most accurate) - 2.4GB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_h_4b8939.pth\r\n\r\n# ViT-L (medium) - 1.2GB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_l_0b3195.pth\r\n\r\n# ViT-B (smallest, fastest) - 375MB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_b_01ec64.pth\r\n```\r\n\r\n### Basic usage with SamPredictor\r\n\r\n```python\r\nimport numpy as np\r\nfrom segment_anything import sam_model_registry, SamPredictor\r\n\r\n# Load model\r\nsam = sam_model_registry[\"vit_h\"](checkpoint=\"sam_vit_h_4b8939.pth\")\r\nsam.to(device=\"cuda\")\r\n\r\n# Create predictor\r\npredictor = SamPredictor(sam)\r\n\r\n# Set image (computes embeddings once)\r\nimage = cv2.imread(\"image.jpg\")\r\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\r\npredictor.set_image(image)\r\n\r\n# Predict with point prompts\r\ninput_point = np.array([[500, 375]])  # (x, y) coordinates\r\ninput_label = np.array([1])  # 1 = foreground, 0 = background\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_point,\r\n    point_labels=input_label,\r\n    multimask_output=True  # Returns 3 mask options\r\n)\r\n\r\n# Select best mask\r\nbest_mask = masks[np.argmax(scores)]\r\n```\r\n\r\n### HuggingFace Transformers\r\n\r\n```python\r\nimport torch\r\nfrom PIL import Image\r\nfrom transformers import SamModel, SamProcessor\r\n\r\n# Load model and processor\r\nmodel = SamModel.from_pretrained(\"facebook\u002Fsam-vit-huge\")\r\nprocessor = SamProcessor.from_pretrained(\"facebook\u002Fsam-vit-huge\")\r\nmodel.to(\"cuda\")\r\n\r\n# Process image with point prompt\r\nimage = Image.open(\"image.jpg\")\r\ninput_points = [[[450, 600]]]  # Batch of points\r\n\r\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\")\r\ninputs = {k: v.to(\"cuda\") for k, v in inputs.items()}\r\n\r\n# Generate masks\r\nwith torch.no_grad():\r\n    outputs = model(**inputs)\r\n\r\n# Post-process masks to original size\r\nmasks = processor.image_processor.post_process_masks(\r\n    outputs.pred_masks.cpu(),\r\n    inputs[\"original_sizes\"].cpu(),\r\n    inputs[\"reshaped_input_sizes\"].cpu()\r\n)\r\n```\r\n\r\n## Core concepts\r\n\r\n### Model architecture\r\n\r\n\u003C!-- ascii-guard-ignore -->\r\n```\r\nSAM Architecture:\r\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\r\n│  Image Encoder  │────▶│ Prompt Encoder  │────▶│  Mask Decoder   │\r\n│     (ViT)       │     │ (Points\u002FBoxes)  │     │ (Transformer)   │\r\n└─────────────────┘     └─────────────────┘     └─────────────────┘\r\n        │                       │                       │\r\n   Image Embeddings      Prompt Embeddings         Masks + IoU\r\n   (computed once)       (per prompt)             predictions\r\n```\r\n\u003C!-- ascii-guard-ignore-end -->\r\n\r\n### Model variants\r\n\r\n| Model | Checkpoint | Size | Speed | Accuracy |\r\n|-------|------------|------|-------|----------|\r\n| ViT-H | `vit_h` | 2.4 GB | Slowest | Best |\r\n| ViT-L | `vit_l` | 1.2 GB | Medium | Good |\r\n| ViT-B | `vit_b` | 375 MB | Fastest | Good |\r\n\r\n### Prompt types\r\n\r\n| Prompt | Description | Use Case |\r\n|--------|-------------|----------|\r\n| Point (foreground) | Click on object | Single object selection |\r\n| Point (background) | Click outside object | Exclude regions |\r\n| Bounding box | Rectangle around object | Larger objects |\r\n| Previous mask | Low-res mask input | Iterative refinement |\r\n\r\n## Interactive segmentation\r\n\r\n### Point prompts\r\n\r\n```python\r\n# Single foreground point\r\ninput_point = np.array([[500, 375]])\r\ninput_label = np.array([1])\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_point,\r\n    point_labels=input_label,\r\n    multimask_output=True\r\n)\r\n\r\n# Multiple points (foreground + background)\r\ninput_points = np.array([[500, 375], [600, 400], [450, 300]])\r\ninput_labels = np.array([1, 1, 0])  # 2 foreground, 1 background\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_points,\r\n    point_labels=input_labels,\r\n    multimask_output=False  # Single mask when prompts are clear\r\n)\r\n```\r\n\r\n### Box prompts\r\n\r\n```python\r\n# Bounding box [x1, y1, x2, y2]\r\ninput_box = np.array([425, 600, 700, 875])\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    box=input_box,\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n### Combined prompts\r\n\r\n```python\r\n# Box + points for precise control\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375]]),\r\n    point_labels=np.array([1]),\r\n    box=np.array([400, 300, 700, 600]),\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n### Iterative refinement\r\n\r\n```python\r\n# Initial prediction\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375]]),\r\n    point_labels=np.array([1]),\r\n    multimask_output=True\r\n)\r\n\r\n# Refine with additional point using previous mask\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375], [550, 400]]),\r\n    point_labels=np.array([1, 0]),  # Add background point\r\n    mask_input=logits[np.argmax(scores)][None, :, :],  # Use best mask\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n## Automatic mask generation\r\n\r\n### Basic automatic segmentation\r\n\r\n```python\r\nfrom segment_anything import SamAutomaticMaskGenerator\r\n\r\n# Create generator\r\nmask_generator = SamAutomaticMaskGenerator(sam)\r\n\r\n# Generate all masks\r\nmasks = mask_generator.generate(image)\r\n\r\n# Each mask contains:\r\n# - segmentation: binary mask\r\n# - bbox: [x, y, w, h]\r\n# - area: pixel count\r\n# - predicted_iou: quality score\r\n# - stability_score: robustness score\r\n# - point_coords: generating point\r\n```\r\n\r\n### Customized generation\r\n\r\n```python\r\nmask_generator = SamAutomaticMaskGenerator(\r\n    model=sam,\r\n    points_per_side=32,          # Grid density (more = more masks)\r\n    pred_iou_thresh=0.88,        # Quality threshold\r\n    stability_score_thresh=0.95,  # Stability threshold\r\n    crop_n_layers=1,             # Multi-scale crops\r\n    crop_n_points_downscale_factor=2,\r\n    min_mask_region_area=100,    # Remove tiny masks\r\n)\r\n\r\nmasks = mask_generator.generate(image)\r\n```\r\n\r\n### Filtering masks\r\n\r\n```python\r\n# Sort by area (largest first)\r\nmasks = sorted(masks, key=lambda x: x['area'], reverse=True)\r\n\r\n# Filter by predicted IoU\r\nhigh_quality = [m for m in masks if m['predicted_iou'] > 0.9]\r\n\r\n# Filter by stability score\r\nstable_masks = [m for m in masks if m['stability_score'] > 0.95]\r\n```\r\n\r\n## Batched inference\r\n\r\n### Multiple images\r\n\r\n```python\r\n# Process multiple images efficiently\r\nimages = [cv2.imread(f\"image_{i}.jpg\") for i in range(10)]\r\n\r\nall_masks = []\r\nfor image in images:\r\n    predictor.set_image(image)\r\n    masks, _, _ = predictor.predict(\r\n        point_coords=np.array([[500, 375]]),\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n    all_masks.append(masks)\r\n```\r\n\r\n### Multiple prompts per image\r\n\r\n```python\r\n# Process multiple prompts efficiently (one image encoding)\r\npredictor.set_image(image)\r\n\r\n# Batch of point prompts\r\npoints = [\r\n    np.array([[100, 100]]),\r\n    np.array([[200, 200]]),\r\n    np.array([[300, 300]])\r\n]\r\n\r\nall_masks = []\r\nfor point in points:\r\n    masks, scores, _ = predictor.predict(\r\n        point_coords=point,\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n    all_masks.append(masks[np.argmax(scores)])\r\n```\r\n\r\n## ONNX deployment\r\n\r\n### Export model\r\n\r\n```bash\r\npython scripts\u002Fexport_onnx_model.py \\\r\n    --checkpoint sam_vit_h_4b8939.pth \\\r\n    --model-type vit_h \\\r\n    --output sam_onnx.onnx \\\r\n    --return-single-mask\r\n```\r\n\r\n### Use ONNX model\r\n\r\n```python\r\nimport onnxruntime\r\n\r\n# Load ONNX model\r\nort_session = onnxruntime.InferenceSession(\"sam_onnx.onnx\")\r\n\r\n# Run inference (image embeddings computed separately)\r\nmasks = ort_session.run(\r\n    None,\r\n    {\r\n        \"image_embeddings\": image_embeddings,\r\n        \"point_coords\": point_coords,\r\n        \"point_labels\": point_labels,\r\n        \"mask_input\": np.zeros((1, 1, 256, 256), dtype=np.float32),\r\n        \"has_mask_input\": np.array([0], dtype=np.float32),\r\n        \"orig_im_size\": np.array([h, w], dtype=np.float32)\r\n    }\r\n)\r\n```\r\n\r\n## Common workflows\r\n\r\n### Workflow 1: Annotation tool\r\n\r\n```python\r\nimport cv2\r\n\r\n# Load model\r\npredictor = SamPredictor(sam)\r\npredictor.set_image(image)\r\n\r\ndef on_click(event, x, y, flags, param):\r\n    if event == cv2.EVENT_LBUTTONDOWN:\r\n        # Foreground point\r\n        masks, scores, _ = predictor.predict(\r\n            point_coords=np.array([[x, y]]),\r\n            point_labels=np.array([1]),\r\n            multimask_output=True\r\n        )\r\n        # Display best mask\r\n        display_mask(masks[np.argmax(scores)])\r\n```\r\n\r\n### Workflow 2: Object extraction\r\n\r\n```python\r\ndef extract_object(image, point):\r\n    \"\"\"Extract object at point with transparent background.\"\"\"\r\n    predictor.set_image(image)\r\n\r\n    masks, scores, _ = predictor.predict(\r\n        point_coords=np.array([point]),\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n\r\n    best_mask = masks[np.argmax(scores)]\r\n\r\n    # Create RGBA output\r\n    rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)\r\n    rgba[:, :, :3] = image\r\n    rgba[:, :, 3] = best_mask * 255\r\n\r\n    return rgba\r\n```\r\n\r\n### Workflow 3: Medical image segmentation\r\n\r\n```python\r\n# Process medical images (grayscale to RGB)\r\nmedical_image = cv2.imread(\"scan.png\", cv2.IMREAD_GRAYSCALE)\r\nrgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)\r\n\r\npredictor.set_image(rgb_image)\r\n\r\n# Segment region of interest\r\nmasks, scores, _ = predictor.predict(\r\n    box=np.array([x1, y1, x2, y2]),  # ROI bounding box\r\n    multimask_output=True\r\n)\r\n```\r\n\r\n## Output format\r\n\r\n### Mask data structure\r\n\r\n```python\r\n# SamAutomaticMaskGenerator output\r\n{\r\n    \"segmentation\": np.ndarray,  # H×W binary mask\r\n    \"bbox\": [x, y, w, h],        # Bounding box\r\n    \"area\": int,                 # Pixel count\r\n    \"predicted_iou\": float,      # 0-1 quality score\r\n    \"stability_score\": float,    # 0-1 robustness score\r\n    \"crop_box\": [x, y, w, h],    # Generation crop region\r\n    \"point_coords\": [[x, y]],    # Input point\r\n}\r\n```\r\n\r\n### COCO RLE format\r\n\r\n```python\r\nfrom pycocotools import mask as mask_utils\r\n\r\n# Encode mask to RLE\r\nrle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))\r\nrle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\r\n\r\n# Decode RLE to mask\r\ndecoded_mask = mask_utils.decode(rle)\r\n```\r\n\r\n## Performance optimization\r\n\r\n### GPU memory\r\n\r\n```python\r\n# Use smaller model for limited VRAM\r\nsam = sam_model_registry[\"vit_b\"](checkpoint=\"sam_vit_b_01ec64.pth\")\r\n\r\n# Process images in batches\r\n# Clear CUDA cache between large batches\r\ntorch.cuda.empty_cache()\r\n```\r\n\r\n### Speed optimization\r\n\r\n```python\r\n# Use half precision\r\nsam = sam.half()\r\n\r\n# Reduce points for automatic generation\r\nmask_generator = SamAutomaticMaskGenerator(\r\n    model=sam,\r\n    points_per_side=16,  # Default is 32\r\n)\r\n\r\n# Use ONNX for deployment\r\n# Export with --return-single-mask for faster inference\r\n```\r\n\r\n## Common issues\r\n\r\n| Issue | Solution |\r\n|-------|----------|\r\n| Out of memory | Use ViT-B model, reduce image size |\r\n| Slow inference | Use ViT-B, reduce points_per_side |\r\n| Poor mask quality | Try different prompts, use box + points |\r\n| Edge artifacts | Use stability_score filtering |\r\n| Small objects missed | Increase points_per_side |\r\n\r\n## References\r\n\r\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Batching, fine-tuning, integration\r\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\r\n\r\n## Resources\r\n\r\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything\r\n- **Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.02643\r\n- **Demo**: https:\u002F\u002Fsegment-anything.com\r\n- **SAM 2 (Video)**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2\r\n- **HuggingFace**: https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fsam-vit-huge\r\n",{"name":5,"author":44,"license":15,"version":45,"metadata":46,"platforms":53,"description":7,"dependencies":54},"Orchestra Research","1.0.0",{"hermes":47},{"tags":48},[49,50,33,51,52],"Multimodal","Image Segmentation","SAM","Zero-Shot",[28,27,30],[55,56,57],"segment-anything","transformers>=4.30.0","torch>=1.7.0","main","0d1434f89f4bc6b07e9d272accd8c1013768820f","archive\u002Fskill-md-reviewed\u002Fsha256\u002F19\u002F198715c51b662f92e21fcdd4f8762427e9277827a0dad0fcfe3314264a3f845d",0,"https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills","https:\u002F\u002Fgithub.com\u002Fzapabob","The Open University of JAPAN　2019-now\r\n\r\nTokyo University of Science　2007-2010","2026-10-26 16:29:01",{"source":8,"skillName":5,"commands":67,"prompt":77,"zipAvailable":78},[68,71,74],{"tool":69,"command":70},"npx","npx skills add zapabob\u002FSkills --skill segment-anything-model",{"tool":72,"command":73},"bunx","bunx skills add zapabob\u002FSkills --skill segment-anything-model",{"tool":75,"command":76},"pnpm","pnpm dlx skills add zapabob\u002FSkills --skill segment-anything-model","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Farchive\u002Fskill-md-reviewed\u002Fsha256\u002F19\u002F198715c51b662f92e21fcdd4f8762427e9277827a0dad0fcfe3314264a3f845d\u002FSKILL.md",true,[],[81,114,149,171,198,215],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":98,"publishedAt":39},"01M6GJTE1GESGGF9K028Y7753E","xurl","X\u002FTwitter via xurl CLI: post, search, DM, media, v2 API.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fxurl\u002Fvariants\u002Fhermes-agent\u002FSKILL.md",[87,90,93],{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":91,"categoryName":92,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":94,"categoryName":95,"taxonomyType":96,"taxonomyTypeDesc":97},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[83,99,27,100,101,28,30,102,103,104,105,106,107,108,109,110,58,111,112,113],"openclaw","cli","github","official","search","https","social-media","api","Media","hermes","hermes agent","twitter","social","via","post",{"skillNo":115,"name":116,"nameZh":6,"description":117,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":118,"categories":119,"keywords":131,"publishedAt":39},"01M6GJT8S1PXQ8WPXT1HSNBWH2","writing-plans","Write implementation plans: bite-sized tasks, paths, code.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fwriting-plans\u002Fvariants\u002Fhermes-agent\u002FSKILL.md",[120,123,126,127,128],{"categoryNo":121,"categoryName":122,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":124,"categoryName":125,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":91,"categoryName":92,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":94,"categoryName":95,"taxonomyType":96,"taxonomyTypeDesc":97},{"categoryNo":129,"categoryName":130,"taxonomyType":96,"taxonomyTypeDesc":97},"01JBSEEDCAT000000000000036","内容创作者",[132,133,134,135,27,136,28,137,30,138,139,140,141,142,143,144,145,146,147,148],"writing","writing plans","plans","design","review","workflow","documentation","tasks","development","code","planning","code-review","subagent","write","superpowers","paths","test-driven-development",{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":156,"publishedAt":39},"01M6GJST0H5X7WHBJ3BZK0DVKH","whisper","OpenAI's general-purpose speech recognition model. Supports 99 languages, transcription, translation to English, and language identification. Six model sizes from tiny (39M params) to large (1550M params). Use for speech-to-text, podcast transcription, or multilingual audio processing. Best for robust, multilingual ASR.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fwhisper\u002Fvariants\u002Fhermes-agent-optional\u002FSKILL.md",[155],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[151,27,28,157,158,31,34,159,160,161,24,162,163,164,165,166,167,168,169,170],"openai","speech","ASR","Audio","text","speech recognition","use","general","multilingual","translation","english","identification","processing","Audio Processing",{"skillNo":172,"name":173,"nameZh":6,"description":174,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":175,"categories":176,"keywords":180,"publishedAt":39},"01M6GJSMRQ8EETZTYMG72P70EA","weights-and-biases","W&B: log ML experiments, sweeps, model registry, dashboards.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fweights-and-biases\u002Fvariants\u002Fhermes-agent\u002FSKILL.md",[177],{"categoryNo":178,"categoryName":179,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化",[181,27,182,183,30,184,185,186,187,188,189,24,190,191,192,193,194,195,196,197],"weights and biases","tensorflow","experiments","pytorch","visualization","registry","real-time","tuning","dashboards","experiment","log","time","Experiment Tracking","Model Registry","Hyperparameter Tuning","WandB","Real-Time Visualization",{"skillNo":199,"name":200,"nameZh":6,"description":201,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":202,"categories":203,"keywords":205,"publishedAt":39},"01M6GJS19KDPRY112JTR326S06","watchers","Poll RSS, JSON APIs, and GitHub with watermark dedup.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fwatchers\u002Fvariants\u002Fhermes-agent-optional\u002FSKILL.md",[204],{"categoryNo":91,"categoryName":92,"taxonomyType":21,"taxonomyTypeDesc":22},[200,27,206,101,28,207,208,209,210,211,212,108,109,213,214],"terminal","automation","http","monitoring","cron","json","rss","devops","with",{"skillNo":216,"name":217,"nameZh":6,"description":218,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":219,"categories":220,"keywords":223,"publishedAt":39},"01M6GJRHJD2A6Q830SWF9W48BC","serving-llms-vllm","vLLM: high-throughput LLM serving, OpenAI API, quantization.","https:\u002F\u002Fgithub.com\u002Fzapabob\u002FSkills\u002Fblob\u002Fmain\u002Fregistry\u002Fskills\u002Fvllm\u002Fvariants\u002Fhermes-agent\u002FSKILL.md",[221,222],{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":94,"categoryName":95,"taxonomyType":96,"taxonomyTypeDesc":97},[224,225,226,28,29,157,227,228,229,34,230,106,231,232,233,234,235,236,237,238],"llms","vLLM","serving llms vllm","openai api","tensor","Inference Serving","Production","llm serving","High Throughput","PagedAttention","Continuous Batching","Quantization","Tensor Parallelism","high-throughput","throughput"]