[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7F7BEDC82VEPC7MGVYMEASG":3,"skill-same:01M7F7BEDC82VEPC7MGVYMEASG":74},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":42,"commitSha":43,"skillDirectory":44,"forksCount":14,"repositoryHtmlUrl":45,"repositoryHomepageUrl":6,"ownerHtmlUrl":46,"ownerBlogUrl":47,"ownerDescription":48,"updatedAt":49,"install":50,"resources":63},"01M7F7BEDC82VEPC7MGVYMEASG","segment-anything-model",null,"SAM: zero-shot image segmentation via points, boxes, masks.","chaorenex1\u002Fhermes-home-content","chaorenex1","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F23194925?v=4","01M7F5NA1D1D02R2MEMYQN2NZT","chan chan","01M7F5NA20KTHMTY2015CYM1TZ",0,"MIT","https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Fmodels\u002Fsegment-anything\u002FSKILL.md",[],[],"2026-11-16 10:05:34",false,"---\r\nname: segment-anything-model\r\ndescription: \"SAM: zero-shot image segmentation via points, boxes, masks.\"\r\nversion: 1.0.0\r\nauthor: Orchestra Research\r\nlicense: MIT\r\ndependencies: [segment-anything, transformers>=4.30.0, torch>=1.7.0]\r\nplatforms: [linux, macos, windows]\r\nmetadata:\r\n  hermes:\r\n    tags: [Multimodal, Image Segmentation, Computer Vision, SAM, Zero-Shot]\r\n\r\n---\r\n\r\n# Segment Anything Model (SAM)\r\n\r\nComprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.\r\n\r\n## When to use SAM\r\n\r\n**Use SAM when:**\r\n- Need to segment any object in images without task-specific training\r\n- Building interactive annotation tools with point\u002Fbox prompts\r\n- Generating training data for other vision models\r\n- Need zero-shot transfer to new image domains\r\n- Building object detection\u002Fsegmentation pipelines\r\n- Processing medical, satellite, or domain-specific images\r\n\r\n**Key features:**\r\n- **Zero-shot segmentation**: Works on any image domain without fine-tuning\r\n- **Flexible prompts**: Points, bounding boxes, or previous masks\r\n- **Automatic segmentation**: Generate all object masks automatically\r\n- **High quality**: Trained on 1.1 billion masks from 11 million images\r\n- **Multiple model sizes**: ViT-B (fastest), ViT-L, ViT-H (most accurate)\r\n- **ONNX export**: Deploy in browsers and edge devices\r\n\r\n**Use alternatives instead:**\r\n- **YOLO\u002FDetectron2**: For real-time object detection with classes\r\n- **Mask2Former**: For semantic\u002Fpanoptic segmentation with categories\r\n- **GroundingDINO + SAM**: For text-prompted segmentation\r\n- **SAM 2**: For video segmentation tasks\r\n\r\n## Quick start\r\n\r\n### Installation\r\n\r\n```bash\r\n# From GitHub\r\npip install git+https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything.git\r\n\r\n# Optional dependencies\r\npip install opencv-python pycocotools matplotlib\r\n\r\n# Or use HuggingFace transformers\r\npip install transformers\r\n```\r\n\r\n### Download checkpoints\r\n\r\n```bash\r\n# ViT-H (largest, most accurate) - 2.4GB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_h_4b8939.pth\r\n\r\n# ViT-L (medium) - 1.2GB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_l_0b3195.pth\r\n\r\n# ViT-B (smallest, fastest) - 375MB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_b_01ec64.pth\r\n```\r\n\r\n### Basic usage with SamPredictor\r\n\r\n```python\r\nimport numpy as np\r\nfrom segment_anything import sam_model_registry, SamPredictor\r\n\r\n# Load model\r\nsam = sam_model_registry[\"vit_h\"](checkpoint=\"sam_vit_h_4b8939.pth\")\r\nsam.to(device=\"cuda\")\r\n\r\n# Create predictor\r\npredictor = SamPredictor(sam)\r\n\r\n# Set image (computes embeddings once)\r\nimage = cv2.imread(\"image.jpg\")\r\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\r\npredictor.set_image(image)\r\n\r\n# Predict with point prompts\r\ninput_point = np.array([[500, 375]])  # (x, y) coordinates\r\ninput_label = np.array([1])  # 1 = foreground, 0 = background\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_point,\r\n    point_labels=input_label,\r\n    multimask_output=True  # Returns 3 mask options\r\n)\r\n\r\n# Select best mask\r\nbest_mask = masks[np.argmax(scores)]\r\n```\r\n\r\n### HuggingFace Transformers\r\n\r\n```python\r\nimport torch\r\nfrom PIL import Image\r\nfrom transformers import SamModel, SamProcessor\r\n\r\n# Load model and processor\r\nmodel = SamModel.from_pretrained(\"facebook\u002Fsam-vit-huge\")\r\nprocessor = SamProcessor.from_pretrained(\"facebook\u002Fsam-vit-huge\")\r\nmodel.to(\"cuda\")\r\n\r\n# Process image with point prompt\r\nimage = Image.open(\"image.jpg\")\r\ninput_points = [[[450, 600]]]  # Batch of points\r\n\r\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\")\r\ninputs = {k: v.to(\"cuda\") for k, v in inputs.items()}\r\n\r\n# Generate masks\r\nwith torch.no_grad():\r\n    outputs = model(**inputs)\r\n\r\n# Post-process masks to original size\r\nmasks = processor.image_processor.post_process_masks(\r\n    outputs.pred_masks.cpu(),\r\n    inputs[\"original_sizes\"].cpu(),\r\n    inputs[\"reshaped_input_sizes\"].cpu()\r\n)\r\n```\r\n\r\n## Core concepts\r\n\r\n### Model architecture\r\n\r\n\u003C!-- ascii-guard-ignore -->\r\n```\r\nSAM Architecture:\r\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\r\n│  Image Encoder  │────▶│ Prompt Encoder  │────▶│  Mask Decoder   │\r\n│     (ViT)       │     │ (Points\u002FBoxes)  │     │ (Transformer)   │\r\n└─────────────────┘     └─────────────────┘     └─────────────────┘\r\n        │                       │                       │\r\n   Image Embeddings      Prompt Embeddings         Masks + IoU\r\n   (computed once)       (per prompt)             predictions\r\n```\r\n\u003C!-- ascii-guard-ignore-end -->\r\n\r\n### Model variants\r\n\r\n| Model | Checkpoint | Size | Speed | Accuracy |\r\n|-------|------------|------|-------|----------|\r\n| ViT-H | `vit_h` | 2.4 GB | Slowest | Best |\r\n| ViT-L | `vit_l` | 1.2 GB | Medium | Good |\r\n| ViT-B | `vit_b` | 375 MB | Fastest | Good |\r\n\r\n### Prompt types\r\n\r\n| Prompt | Description | Use Case |\r\n|--------|-------------|----------|\r\n| Point (foreground) | Click on object | Single object selection |\r\n| Point (background) | Click outside object | Exclude regions |\r\n| Bounding box | Rectangle around object | Larger objects |\r\n| Previous mask | Low-res mask input | Iterative refinement |\r\n\r\n## Interactive segmentation\r\n\r\n### Point prompts\r\n\r\n```python\r\n# Single foreground point\r\ninput_point = np.array([[500, 375]])\r\ninput_label = np.array([1])\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_point,\r\n    point_labels=input_label,\r\n    multimask_output=True\r\n)\r\n\r\n# Multiple points (foreground + background)\r\ninput_points = np.array([[500, 375], [600, 400], [450, 300]])\r\ninput_labels = np.array([1, 1, 0])  # 2 foreground, 1 background\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_points,\r\n    point_labels=input_labels,\r\n    multimask_output=False  # Single mask when prompts are clear\r\n)\r\n```\r\n\r\n### Box prompts\r\n\r\n```python\r\n# Bounding box [x1, y1, x2, y2]\r\ninput_box = np.array([425, 600, 700, 875])\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    box=input_box,\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n### Combined prompts\r\n\r\n```python\r\n# Box + points for precise control\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375]]),\r\n    point_labels=np.array([1]),\r\n    box=np.array([400, 300, 700, 600]),\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n### Iterative refinement\r\n\r\n```python\r\n# Initial prediction\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375]]),\r\n    point_labels=np.array([1]),\r\n    multimask_output=True\r\n)\r\n\r\n# Refine with additional point using previous mask\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375], [550, 400]]),\r\n    point_labels=np.array([1, 0]),  # Add background point\r\n    mask_input=logits[np.argmax(scores)][None, :, :],  # Use best mask\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n## Automatic mask generation\r\n\r\n### Basic automatic segmentation\r\n\r\n```python\r\nfrom segment_anything import SamAutomaticMaskGenerator\r\n\r\n# Create generator\r\nmask_generator = SamAutomaticMaskGenerator(sam)\r\n\r\n# Generate all masks\r\nmasks = mask_generator.generate(image)\r\n\r\n# Each mask contains:\r\n# - segmentation: binary mask\r\n# - bbox: [x, y, w, h]\r\n# - area: pixel count\r\n# - predicted_iou: quality score\r\n# - stability_score: robustness score\r\n# - point_coords: generating point\r\n```\r\n\r\n### Customized generation\r\n\r\n```python\r\nmask_generator = SamAutomaticMaskGenerator(\r\n    model=sam,\r\n    points_per_side=32,          # Grid density (more = more masks)\r\n    pred_iou_thresh=0.88,        # Quality threshold\r\n    stability_score_thresh=0.95,  # Stability threshold\r\n    crop_n_layers=1,             # Multi-scale crops\r\n    crop_n_points_downscale_factor=2,\r\n    min_mask_region_area=100,    # Remove tiny masks\r\n)\r\n\r\nmasks = mask_generator.generate(image)\r\n```\r\n\r\n### Filtering masks\r\n\r\n```python\r\n# Sort by area (largest first)\r\nmasks = sorted(masks, key=lambda x: x['area'], reverse=True)\r\n\r\n# Filter by predicted IoU\r\nhigh_quality = [m for m in masks if m['predicted_iou'] > 0.9]\r\n\r\n# Filter by stability score\r\nstable_masks = [m for m in masks if m['stability_score'] > 0.95]\r\n```\r\n\r\n## Batched inference\r\n\r\n### Multiple images\r\n\r\n```python\r\n# Process multiple images efficiently\r\nimages = [cv2.imread(f\"image_{i}.jpg\") for i in range(10)]\r\n\r\nall_masks = []\r\nfor image in images:\r\n    predictor.set_image(image)\r\n    masks, _, _ = predictor.predict(\r\n        point_coords=np.array([[500, 375]]),\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n    all_masks.append(masks)\r\n```\r\n\r\n### Multiple prompts per image\r\n\r\n```python\r\n# Process multiple prompts efficiently (one image encoding)\r\npredictor.set_image(image)\r\n\r\n# Batch of point prompts\r\npoints = [\r\n    np.array([[100, 100]]),\r\n    np.array([[200, 200]]),\r\n    np.array([[300, 300]])\r\n]\r\n\r\nall_masks = []\r\nfor point in points:\r\n    masks, scores, _ = predictor.predict(\r\n        point_coords=point,\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n    all_masks.append(masks[np.argmax(scores)])\r\n```\r\n\r\n## ONNX deployment\r\n\r\n### Export model\r\n\r\n```bash\r\npython scripts\u002Fexport_onnx_model.py \\\r\n    --checkpoint sam_vit_h_4b8939.pth \\\r\n    --model-type vit_h \\\r\n    --output sam_onnx.onnx \\\r\n    --return-single-mask\r\n```\r\n\r\n### Use ONNX model\r\n\r\n```python\r\nimport onnxruntime\r\n\r\n# Load ONNX model\r\nort_session = onnxruntime.InferenceSession(\"sam_onnx.onnx\")\r\n\r\n# Run inference (image embeddings computed separately)\r\nmasks = ort_session.run(\r\n    None,\r\n    {\r\n        \"image_embeddings\": image_embeddings,\r\n        \"point_coords\": point_coords,\r\n        \"point_labels\": point_labels,\r\n        \"mask_input\": np.zeros((1, 1, 256, 256), dtype=np.float32),\r\n        \"has_mask_input\": np.array([0], dtype=np.float32),\r\n        \"orig_im_size\": np.array([h, w], dtype=np.float32)\r\n    }\r\n)\r\n```\r\n\r\n## Common workflows\r\n\r\n### Workflow 1: Annotation tool\r\n\r\n```python\r\nimport cv2\r\n\r\n# Load model\r\npredictor = SamPredictor(sam)\r\npredictor.set_image(image)\r\n\r\ndef on_click(event, x, y, flags, param):\r\n    if event == cv2.EVENT_LBUTTONDOWN:\r\n        # Foreground point\r\n        masks, scores, _ = predictor.predict(\r\n            point_coords=np.array([[x, y]]),\r\n            point_labels=np.array([1]),\r\n            multimask_output=True\r\n        )\r\n        # Display best mask\r\n        display_mask(masks[np.argmax(scores)])\r\n```\r\n\r\n### Workflow 2: Object extraction\r\n\r\n```python\r\ndef extract_object(image, point):\r\n    \"\"\"Extract object at point with transparent background.\"\"\"\r\n    predictor.set_image(image)\r\n\r\n    masks, scores, _ = predictor.predict(\r\n        point_coords=np.array([point]),\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n\r\n    best_mask = masks[np.argmax(scores)]\r\n\r\n    # Create RGBA output\r\n    rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)\r\n    rgba[:, :, :3] = image\r\n    rgba[:, :, 3] = best_mask * 255\r\n\r\n    return rgba\r\n```\r\n\r\n### Workflow 3: Medical image segmentation\r\n\r\n```python\r\n# Process medical images (grayscale to RGB)\r\nmedical_image = cv2.imread(\"scan.png\", cv2.IMREAD_GRAYSCALE)\r\nrgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)\r\n\r\npredictor.set_image(rgb_image)\r\n\r\n# Segment region of interest\r\nmasks, scores, _ = predictor.predict(\r\n    box=np.array([x1, y1, x2, y2]),  # ROI bounding box\r\n    multimask_output=True\r\n)\r\n```\r\n\r\n## Output format\r\n\r\n### Mask data structure\r\n\r\n```python\r\n# SamAutomaticMaskGenerator output\r\n{\r\n    \"segmentation\": np.ndarray,  # H×W binary mask\r\n    \"bbox\": [x, y, w, h],        # Bounding box\r\n    \"area\": int,                 # Pixel count\r\n    \"predicted_iou\": float,      # 0-1 quality score\r\n    \"stability_score\": float,    # 0-1 robustness score\r\n    \"crop_box\": [x, y, w, h],    # Generation crop region\r\n    \"point_coords\": [[x, y]],    # Input point\r\n}\r\n```\r\n\r\n### COCO RLE format\r\n\r\n```python\r\nfrom pycocotools import mask as mask_utils\r\n\r\n# Encode mask to RLE\r\nrle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))\r\nrle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\r\n\r\n# Decode RLE to mask\r\ndecoded_mask = mask_utils.decode(rle)\r\n```\r\n\r\n## Performance optimization\r\n\r\n### GPU memory\r\n\r\n```python\r\n# Use smaller model for limited VRAM\r\nsam = sam_model_registry[\"vit_b\"](checkpoint=\"sam_vit_b_01ec64.pth\")\r\n\r\n# Process images in batches\r\n# Clear CUDA cache between large batches\r\ntorch.cuda.empty_cache()\r\n```\r\n\r\n### Speed optimization\r\n\r\n```python\r\n# Use half precision\r\nsam = sam.half()\r\n\r\n# Reduce points for automatic generation\r\nmask_generator = SamAutomaticMaskGenerator(\r\n    model=sam,\r\n    points_per_side=16,  # Default is 32\r\n)\r\n\r\n# Use ONNX for deployment\r\n# Export with --return-single-mask for faster inference\r\n```\r\n\r\n## Common issues\r\n\r\n| Issue | Solution |\r\n|-------|----------|\r\n| Out of memory | Use ViT-B model, reduce image size |\r\n| Slow inference | Use ViT-B, reduce points_per_side |\r\n| Poor mask quality | Try different prompts, use box + points |\r\n| Edge artifacts | Use stability_score filtering |\r\n| Small objects missed | Increase points_per_side |\r\n\r\n## References\r\n\r\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Batching, fine-tuning, integration\r\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\r\n\r\n## Resources\r\n\r\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything\r\n- **Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.02643\r\n- **Demo**: https:\u002F\u002Fsegment-anything.com\r\n- **SAM 2 (Video)**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2\r\n- **HuggingFace**: https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fsam-vit-huge\r\n","# Segment Anything Model (SAM)\r\n\r\nComprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.\r\n\r\n## When to use SAM\r\n\r\n**Use SAM when:**\r\n- Need to segment any object in images without task-specific training\r\n- Building interactive annotation tools with point\u002Fbox prompts\r\n- Generating training data for other vision models\r\n- Need zero-shot transfer to new image domains\r\n- Building object detection\u002Fsegmentation pipelines\r\n- Processing medical, satellite, or domain-specific images\r\n\r\n**Key features:**\r\n- **Zero-shot segmentation**: Works on any image domain without fine-tuning\r\n- **Flexible prompts**: Points, bounding boxes, or previous masks\r\n- **Automatic segmentation**: Generate all object masks automatically\r\n- **High quality**: Trained on 1.1 billion masks from 11 million images\r\n- **Multiple model sizes**: ViT-B (fastest), ViT-L, ViT-H (most accurate)\r\n- **ONNX export**: Deploy in browsers and edge devices\r\n\r\n**Use alternatives instead:**\r\n- **YOLO\u002FDetectron2**: For real-time object detection with classes\r\n- **Mask2Former**: For semantic\u002Fpanoptic segmentation with categories\r\n- **GroundingDINO + SAM**: For text-prompted segmentation\r\n- **SAM 2**: For video segmentation tasks\r\n\r\n## Quick start\r\n\r\n### Installation\r\n\r\n```bash\r\n# From GitHub\r\npip install git+https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything.git\r\n\r\n# Optional dependencies\r\npip install opencv-python pycocotools matplotlib\r\n\r\n# Or use HuggingFace transformers\r\npip install transformers\r\n```\r\n\r\n### Download checkpoints\r\n\r\n```bash\r\n# ViT-H (largest, most accurate) - 2.4GB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_h_4b8939.pth\r\n\r\n# ViT-L (medium) - 1.2GB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_l_0b3195.pth\r\n\r\n# ViT-B (smallest, fastest) - 375MB\r\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_b_01ec64.pth\r\n```\r\n\r\n### Basic usage with SamPredictor\r\n\r\n```python\r\nimport numpy as np\r\nfrom segment_anything import sam_model_registry, SamPredictor\r\n\r\n# Load model\r\nsam = sam_model_registry[\"vit_h\"](checkpoint=\"sam_vit_h_4b8939.pth\")\r\nsam.to(device=\"cuda\")\r\n\r\n# Create predictor\r\npredictor = SamPredictor(sam)\r\n\r\n# Set image (computes embeddings once)\r\nimage = cv2.imread(\"image.jpg\")\r\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\r\npredictor.set_image(image)\r\n\r\n# Predict with point prompts\r\ninput_point = np.array([[500, 375]])  # (x, y) coordinates\r\ninput_label = np.array([1])  # 1 = foreground, 0 = background\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_point,\r\n    point_labels=input_label,\r\n    multimask_output=True  # Returns 3 mask options\r\n)\r\n\r\n# Select best mask\r\nbest_mask = masks[np.argmax(scores)]\r\n```\r\n\r\n### HuggingFace Transformers\r\n\r\n```python\r\nimport torch\r\nfrom PIL import Image\r\nfrom transformers import SamModel, SamProcessor\r\n\r\n# Load model and processor\r\nmodel = SamModel.from_pretrained(\"facebook\u002Fsam-vit-huge\")\r\nprocessor = SamProcessor.from_pretrained(\"facebook\u002Fsam-vit-huge\")\r\nmodel.to(\"cuda\")\r\n\r\n# Process image with point prompt\r\nimage = Image.open(\"image.jpg\")\r\ninput_points = [[[450, 600]]]  # Batch of points\r\n\r\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\")\r\ninputs = {k: v.to(\"cuda\") for k, v in inputs.items()}\r\n\r\n# Generate masks\r\nwith torch.no_grad():\r\n    outputs = model(**inputs)\r\n\r\n# Post-process masks to original size\r\nmasks = processor.image_processor.post_process_masks(\r\n    outputs.pred_masks.cpu(),\r\n    inputs[\"original_sizes\"].cpu(),\r\n    inputs[\"reshaped_input_sizes\"].cpu()\r\n)\r\n```\r\n\r\n## Core concepts\r\n\r\n### Model architecture\r\n\r\n\u003C!-- ascii-guard-ignore -->\r\n```\r\nSAM Architecture:\r\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\r\n│  Image Encoder  │────▶│ Prompt Encoder  │────▶│  Mask Decoder   │\r\n│     (ViT)       │     │ (Points\u002FBoxes)  │     │ (Transformer)   │\r\n└─────────────────┘     └─────────────────┘     └─────────────────┘\r\n        │                       │                       │\r\n   Image Embeddings      Prompt Embeddings         Masks + IoU\r\n   (computed once)       (per prompt)             predictions\r\n```\r\n\u003C!-- ascii-guard-ignore-end -->\r\n\r\n### Model variants\r\n\r\n| Model | Checkpoint | Size | Speed | Accuracy |\r\n|-------|------------|------|-------|----------|\r\n| ViT-H | `vit_h` | 2.4 GB | Slowest | Best |\r\n| ViT-L | `vit_l` | 1.2 GB | Medium | Good |\r\n| ViT-B | `vit_b` | 375 MB | Fastest | Good |\r\n\r\n### Prompt types\r\n\r\n| Prompt | Description | Use Case |\r\n|--------|-------------|----------|\r\n| Point (foreground) | Click on object | Single object selection |\r\n| Point (background) | Click outside object | Exclude regions |\r\n| Bounding box | Rectangle around object | Larger objects |\r\n| Previous mask | Low-res mask input | Iterative refinement |\r\n\r\n## Interactive segmentation\r\n\r\n### Point prompts\r\n\r\n```python\r\n# Single foreground point\r\ninput_point = np.array([[500, 375]])\r\ninput_label = np.array([1])\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_point,\r\n    point_labels=input_label,\r\n    multimask_output=True\r\n)\r\n\r\n# Multiple points (foreground + background)\r\ninput_points = np.array([[500, 375], [600, 400], [450, 300]])\r\ninput_labels = np.array([1, 1, 0])  # 2 foreground, 1 background\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=input_points,\r\n    point_labels=input_labels,\r\n    multimask_output=False  # Single mask when prompts are clear\r\n)\r\n```\r\n\r\n### Box prompts\r\n\r\n```python\r\n# Bounding box [x1, y1, x2, y2]\r\ninput_box = np.array([425, 600, 700, 875])\r\n\r\nmasks, scores, logits = predictor.predict(\r\n    box=input_box,\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n### Combined prompts\r\n\r\n```python\r\n# Box + points for precise control\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375]]),\r\n    point_labels=np.array([1]),\r\n    box=np.array([400, 300, 700, 600]),\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n### Iterative refinement\r\n\r\n```python\r\n# Initial prediction\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375]]),\r\n    point_labels=np.array([1]),\r\n    multimask_output=True\r\n)\r\n\r\n# Refine with additional point using previous mask\r\nmasks, scores, logits = predictor.predict(\r\n    point_coords=np.array([[500, 375], [550, 400]]),\r\n    point_labels=np.array([1, 0]),  # Add background point\r\n    mask_input=logits[np.argmax(scores)][None, :, :],  # Use best mask\r\n    multimask_output=False\r\n)\r\n```\r\n\r\n## Automatic mask generation\r\n\r\n### Basic automatic segmentation\r\n\r\n```python\r\nfrom segment_anything import SamAutomaticMaskGenerator\r\n\r\n# Create generator\r\nmask_generator = SamAutomaticMaskGenerator(sam)\r\n\r\n# Generate all masks\r\nmasks = mask_generator.generate(image)\r\n\r\n# Each mask contains:\r\n# - segmentation: binary mask\r\n# - bbox: [x, y, w, h]\r\n# - area: pixel count\r\n# - predicted_iou: quality score\r\n# - stability_score: robustness score\r\n# - point_coords: generating point\r\n```\r\n\r\n### Customized generation\r\n\r\n```python\r\nmask_generator = SamAutomaticMaskGenerator(\r\n    model=sam,\r\n    points_per_side=32,          # Grid density (more = more masks)\r\n    pred_iou_thresh=0.88,        # Quality threshold\r\n    stability_score_thresh=0.95,  # Stability threshold\r\n    crop_n_layers=1,             # Multi-scale crops\r\n    crop_n_points_downscale_factor=2,\r\n    min_mask_region_area=100,    # Remove tiny masks\r\n)\r\n\r\nmasks = mask_generator.generate(image)\r\n```\r\n\r\n### Filtering masks\r\n\r\n```python\r\n# Sort by area (largest first)\r\nmasks = sorted(masks, key=lambda x: x['area'], reverse=True)\r\n\r\n# Filter by predicted IoU\r\nhigh_quality = [m for m in masks if m['predicted_iou'] > 0.9]\r\n\r\n# Filter by stability score\r\nstable_masks = [m for m in masks if m['stability_score'] > 0.95]\r\n```\r\n\r\n## Batched inference\r\n\r\n### Multiple images\r\n\r\n```python\r\n# Process multiple images efficiently\r\nimages = [cv2.imread(f\"image_{i}.jpg\") for i in range(10)]\r\n\r\nall_masks = []\r\nfor image in images:\r\n    predictor.set_image(image)\r\n    masks, _, _ = predictor.predict(\r\n        point_coords=np.array([[500, 375]]),\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n    all_masks.append(masks)\r\n```\r\n\r\n### Multiple prompts per image\r\n\r\n```python\r\n# Process multiple prompts efficiently (one image encoding)\r\npredictor.set_image(image)\r\n\r\n# Batch of point prompts\r\npoints = [\r\n    np.array([[100, 100]]),\r\n    np.array([[200, 200]]),\r\n    np.array([[300, 300]])\r\n]\r\n\r\nall_masks = []\r\nfor point in points:\r\n    masks, scores, _ = predictor.predict(\r\n        point_coords=point,\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n    all_masks.append(masks[np.argmax(scores)])\r\n```\r\n\r\n## ONNX deployment\r\n\r\n### Export model\r\n\r\n```bash\r\npython scripts\u002Fexport_onnx_model.py \\\r\n    --checkpoint sam_vit_h_4b8939.pth \\\r\n    --model-type vit_h \\\r\n    --output sam_onnx.onnx \\\r\n    --return-single-mask\r\n```\r\n\r\n### Use ONNX model\r\n\r\n```python\r\nimport onnxruntime\r\n\r\n# Load ONNX model\r\nort_session = onnxruntime.InferenceSession(\"sam_onnx.onnx\")\r\n\r\n# Run inference (image embeddings computed separately)\r\nmasks = ort_session.run(\r\n    None,\r\n    {\r\n        \"image_embeddings\": image_embeddings,\r\n        \"point_coords\": point_coords,\r\n        \"point_labels\": point_labels,\r\n        \"mask_input\": np.zeros((1, 1, 256, 256), dtype=np.float32),\r\n        \"has_mask_input\": np.array([0], dtype=np.float32),\r\n        \"orig_im_size\": np.array([h, w], dtype=np.float32)\r\n    }\r\n)\r\n```\r\n\r\n## Common workflows\r\n\r\n### Workflow 1: Annotation tool\r\n\r\n```python\r\nimport cv2\r\n\r\n# Load model\r\npredictor = SamPredictor(sam)\r\npredictor.set_image(image)\r\n\r\ndef on_click(event, x, y, flags, param):\r\n    if event == cv2.EVENT_LBUTTONDOWN:\r\n        # Foreground point\r\n        masks, scores, _ = predictor.predict(\r\n            point_coords=np.array([[x, y]]),\r\n            point_labels=np.array([1]),\r\n            multimask_output=True\r\n        )\r\n        # Display best mask\r\n        display_mask(masks[np.argmax(scores)])\r\n```\r\n\r\n### Workflow 2: Object extraction\r\n\r\n```python\r\ndef extract_object(image, point):\r\n    \"\"\"Extract object at point with transparent background.\"\"\"\r\n    predictor.set_image(image)\r\n\r\n    masks, scores, _ = predictor.predict(\r\n        point_coords=np.array([point]),\r\n        point_labels=np.array([1]),\r\n        multimask_output=True\r\n    )\r\n\r\n    best_mask = masks[np.argmax(scores)]\r\n\r\n    # Create RGBA output\r\n    rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)\r\n    rgba[:, :, :3] = image\r\n    rgba[:, :, 3] = best_mask * 255\r\n\r\n    return rgba\r\n```\r\n\r\n### Workflow 3: Medical image segmentation\r\n\r\n```python\r\n# Process medical images (grayscale to RGB)\r\nmedical_image = cv2.imread(\"scan.png\", cv2.IMREAD_GRAYSCALE)\r\nrgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)\r\n\r\npredictor.set_image(rgb_image)\r\n\r\n# Segment region of interest\r\nmasks, scores, _ = predictor.predict(\r\n    box=np.array([x1, y1, x2, y2]),  # ROI bounding box\r\n    multimask_output=True\r\n)\r\n```\r\n\r\n## Output format\r\n\r\n### Mask data structure\r\n\r\n```python\r\n# SamAutomaticMaskGenerator output\r\n{\r\n    \"segmentation\": np.ndarray,  # H×W binary mask\r\n    \"bbox\": [x, y, w, h],        # Bounding box\r\n    \"area\": int,                 # Pixel count\r\n    \"predicted_iou\": float,      # 0-1 quality score\r\n    \"stability_score\": float,    # 0-1 robustness score\r\n    \"crop_box\": [x, y, w, h],    # Generation crop region\r\n    \"point_coords\": [[x, y]],    # Input point\r\n}\r\n```\r\n\r\n### COCO RLE format\r\n\r\n```python\r\nfrom pycocotools import mask as mask_utils\r\n\r\n# Encode mask to RLE\r\nrle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))\r\nrle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\r\n\r\n# Decode RLE to mask\r\ndecoded_mask = mask_utils.decode(rle)\r\n```\r\n\r\n## Performance optimization\r\n\r\n### GPU memory\r\n\r\n```python\r\n# Use smaller model for limited VRAM\r\nsam = sam_model_registry[\"vit_b\"](checkpoint=\"sam_vit_b_01ec64.pth\")\r\n\r\n# Process images in batches\r\n# Clear CUDA cache between large batches\r\ntorch.cuda.empty_cache()\r\n```\r\n\r\n### Speed optimization\r\n\r\n```python\r\n# Use half precision\r\nsam = sam.half()\r\n\r\n# Reduce points for automatic generation\r\nmask_generator = SamAutomaticMaskGenerator(\r\n    model=sam,\r\n    points_per_side=16,  # Default is 32\r\n)\r\n\r\n# Use ONNX for deployment\r\n# Export with --return-single-mask for faster inference\r\n```\r\n\r\n## Common issues\r\n\r\n| Issue | Solution |\r\n|-------|----------|\r\n| Out of memory | Use ViT-B model, reduce image size |\r\n| Slow inference | Use ViT-B, reduce points_per_side |\r\n| Poor mask quality | Try different prompts, use box + points |\r\n| Edge artifacts | Use stability_score filtering |\r\n| Small objects missed | Increase points_per_side |\r\n\r\n## References\r\n\r\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Batching, fine-tuning, integration\r\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\r\n\r\n## Resources\r\n\r\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything\r\n- **Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.02643\r\n- **Demo**: https:\u002F\u002Fsegment-anything.com\r\n- **SAM 2 (Video)**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2\r\n- **HuggingFace**: https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fsam-vit-huge\r\n",{"name":5,"author":24,"license":15,"version":25,"metadata":26,"platforms":34,"description":7,"dependencies":38},"Orchestra Research","1.0.0",{"hermes":27},{"tags":28},[29,30,31,32,33],"Multimodal","Image Segmentation","Computer Vision","SAM","Zero-Shot",[35,36,37],"linux","macos","windows",[39,40,41],"segment-anything","transformers>=4.30.0","torch>=1.7.0","main","a3f3199bb87ec3c91248f68a39d88527e1847ae9","skills\u002Fmlops\u002Fmodels\u002Fsegment-anything","https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content","https:\u002F\u002Fgithub.com\u002Fchaorenex1","https:\u002F\u002Fblog.aduib.com","A programmer whose only hobby in life is coding","2026-10-20 00:34:32",{"source":8,"skillName":5,"commands":51,"prompt":61,"zipAvailable":62},[52,55,58],{"tool":53,"command":54},"npx","npx skills add chaorenex1\u002Fhermes-home-content --skill segment-anything-model",{"tool":56,"command":57},"bunx","bunx skills add chaorenex1\u002Fhermes-home-content --skill segment-anything-model",{"tool":59,"command":60},"pnpm","pnpm dlx skills add chaorenex1\u002Fhermes-home-content --skill segment-anything-model","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Fmodels\u002Fsegment-anything\u002FSKILL.md",true,[64,70],{"path":65,"resourceType":66,"resourceTypeDesc":67,"sizeBytes":68,"sourceHtmlUrl":69},"references\u002Fadvanced-usage.md","REFERENCE","参考文档",16392,"https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Fmodels\u002Fsegment-anything\u002Freferences\u002Fadvanced-usage.md",{"path":71,"resourceType":66,"resourceTypeDesc":67,"sizeBytes":72,"sourceHtmlUrl":73},"references\u002Ftroubleshooting.md",12398,"https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Fmlops\u002Fmodels\u002Fsegment-anything\u002Freferences\u002Ftroubleshooting.md",[75,82,89,96,103,110],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":19},"01M7F8GF32YD5MBQW2TGSRH1YP","web-retrieval-resilience","Web tools fail or search rules break: repair or fall back.","https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Fweb\u002Fweb-retrieval-resilience\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":19},"01M7F8FP46QESQMCZ1KBT3TW3R","blocked-page-recovery","Use when a fetch fails: 403\u002F429, paywall, WAF, bot wall.","https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Fweb\u002Fblocked-page-recovery\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":95,"publishedAt":19},"01M7F8ER6VXHBNVRQBZS7X08SE","verification-loop","Run a six-phase verification of a Claude Code session's work — build, type check, lint, tests with coverage, security grep, and diff review — then produce a PASS\u002FFAIL verification report. Use when verifying work after completing a feature or refactor, before creating a PR, or when quality gates must pass.","https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Fverification-loop\u002FSKILL.md",[],[],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":19},"01M7F8E4BQJ38KVQVM3MT20PYV","unreal-world-builder","开放世界与环境专家——精通 UE5 World Partition、Landscape、程序化植被、HLOD 和大规模关卡流式加载，打造无缝开放世界体验","https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Funreal-engine\u002Funreal-world-builder\u002FSKILL.md",[],[],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":19},"01M7F8D747SQW2FTP7GBSX38BR","unreal-technical-artist","Unreal Engine 视觉管线专家——精通材质编辑器、Niagara 特效、程序化内容生成和 UE5 项目的美术到引擎管线","https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Funreal-engine\u002Funreal-technical-artist\u002FSKILL.md",[],[],{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":114,"categories":115,"keywords":116,"publishedAt":19},"01M7F8CKHYJVZVBTN0321J88T2","unreal-systems-engineer","性能与混合架构专家——精通 C++\u002FBlueprint 边界、Nanite 几何体、Lumen GI 和 Gameplay Ability System，面向 AAA 级 Unreal Engine 项目","https:\u002F\u002Fgithub.com\u002Fchaorenex1\u002Fhermes-home-content\u002Fblob\u002Fmain\u002Fskills\u002Funreal-engine\u002Funreal-systems-engineer\u002FSKILL.md",[],[]]