[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MB4DBMAZA24VCTKB3VD41V":3,"skill-same:01M6MB4DBMAZA24VCTKB3VD41V":97},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":26,"publishedAt":48,"linkOnly":49,"markdownContent":50,"markdownBody":51,"frontMatter":52,"refName":59,"commitSha":60,"skillDirectory":61,"forksCount":62,"repositoryHtmlUrl":63,"repositoryHomepageUrl":64,"ownerHtmlUrl":65,"ownerBlogUrl":66,"ownerDescription":67,"updatedAt":68,"install":69,"resources":82},"01M6MB4DBMAZA24VCTKB3VD41V","segment-anything-model",null,"Foundation model for image segmentation with zero-shot transfer. Use when you need to segment any object in images using points, boxes, or masks as prompts, or automatically generate all object masks in an image.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1554,"MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47],"segment anything","model","segment","segment anything model","images","when","for","object","you","Zero-Shot","boxes","as","image","vision","use","computer","segmentation","Transformers","ZeRO","multimodal","generate","2026-11-10 10:12:30",false,"---\nname: segment-anything-model\ndescription: Foundation model for image segmentation with zero-shot transfer. Use when you need to segment any object in images using points, boxes, or masks as prompts, or automatically generate all object masks in an image.\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: multimodal-and-emerging\n  maintainer: Kalaris Labs\n  tags: Multimodal, Image Segmentation, Computer Vision, SAM, Zero-Shot\n  dependencies: segment-anything, transformers>=4.30.0, torch>=1.7.0\n---\n\n# Segment Anything Model (SAM)\n\nComprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.\n\n## When to use SAM\n\n**Use SAM when:**\n- Need to segment any object in images without task-specific training\n- Building interactive annotation tools with point\u002Fbox prompts\n- Generating training data for other vision models\n- Need zero-shot transfer to new image domains\n- Building object detection\u002Fsegmentation pipelines\n- Processing medical, satellite, or domain-specific images\n\n**Key features:**\n- **Zero-shot segmentation**: Works on any image domain without fine-tuning\n- **Flexible prompts**: Points, bounding boxes, or previous masks\n- **Automatic segmentation**: Generate all object masks automatically\n- **High quality**: Trained on 1.1 billion masks from 11 million images\n- **Multiple model sizes**: ViT-B (fastest), ViT-L, ViT-H (most accurate)\n- **ONNX export**: Deploy in browsers and edge devices\n\n**Use alternatives instead:**\n- **YOLO\u002FDetectron2**: For real-time object detection with classes\n- **Mask2Former**: For semantic\u002Fpanoptic segmentation with categories\n- **GroundingDINO + SAM**: For text-prompted segmentation\n- **SAM 2**: For video segmentation tasks\n\n## Quick start\n\n### Installation\n\n```bash\n# From GitHub\npip install git+https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything.git\n\n# Optional dependencies\npip install opencv-python pycocotools matplotlib\n\n# Or use HuggingFace transformers\npip install transformers\n```\n\n### Download checkpoints\n\n```bash\n# ViT-H (largest, most accurate) - 2.4GB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_h_4b8939.pth\n\n# ViT-L (medium) - 1.2GB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_l_0b3195.pth\n\n# ViT-B (smallest, fastest) - 375MB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_b_01ec64.pth\n```\n\n### Basic usage with SamPredictor\n\n```python\nimport numpy as np\nfrom segment_anything import sam_model_registry, SamPredictor\n\n# Load model\nsam = sam_model_registry[\"vit_h\"](checkpoint=\"sam_vit_h_4b8939.pth\")\nsam.to(device=\"cuda\")\n\n# Create predictor\npredictor = SamPredictor(sam)\n\n# Set image (computes embeddings once)\nimage = cv2.imread(\"image.jpg\")\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\npredictor.set_image(image)\n\n# Predict with point prompts\ninput_point = np.array([[500, 375]])  # (x, y) coordinates\ninput_label = np.array([1])  # 1 = foreground, 0 = background\n\nmasks, scores, logits = predictor.predict(\n    point_coords=input_point,\n    point_labels=input_label,\n    multimask_output=True  # Returns 3 mask options\n)\n\n# Select best mask\nbest_mask = masks[np.argmax(scores)]\n```\n\n### HuggingFace Transformers\n\n```python\nimport torch\nfrom PIL import Image\nfrom transformers import SamModel, SamProcessor\n\n# Load model and processor\nmodel = SamModel.from_pretrained(\"facebook\u002Fsam-vit-huge\")\nprocessor = SamProcessor.from_pretrained(\"facebook\u002Fsam-vit-huge\")\nmodel.to(\"cuda\")\n\n# Process image with point prompt\nimage = Image.open(\"image.jpg\")\ninput_points = [[[450, 600]]]  # Batch of points\n\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\")\ninputs = {k: v.to(\"cuda\") for k, v in inputs.items()}\n\n# Generate masks\nwith torch.no_grad():\n    outputs = model(**inputs)\n\n# Post-process masks to original size\nmasks = processor.image_processor.post_process_masks(\n    outputs.pred_masks.cpu(),\n    inputs[\"original_sizes\"].cpu(),\n    inputs[\"reshaped_input_sizes\"].cpu()\n)\n```\n\n## Core concepts\n\n### Model architecture\n\n```\nSAM Architecture:\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\n│  Image Encoder  │────▶│ Prompt Encoder  │────▶│  Mask Decoder   │\n│     (ViT)       │     │ (Points\u002FBoxes)  │     │ (Transformer)   │\n└─────────────────┘     └─────────────────┘     └─────────────────┘\n        │                       │                       │\n   Image Embeddings      Prompt Embeddings         Masks + IoU\n   (computed once)       (per prompt)             predictions\n```\n\n### Model variants\n\n| Model | Checkpoint | Size | Speed | Accuracy |\n|-------|------------|------|-------|----------|\n| ViT-H | `vit_h` | 2.4 GB | Slowest | Best |\n| ViT-L | `vit_l` | 1.2 GB | Medium | Good |\n| ViT-B | `vit_b` | 375 MB | Fastest | Good |\n\n### Prompt types\n\n| Prompt | Description | Use Case |\n|--------|-------------|----------|\n| Point (foreground) | Click on object | Single object selection |\n| Point (background) | Click outside object | Exclude regions |\n| Bounding box | Rectangle around object | Larger objects |\n| Previous mask | Low-res mask input | Iterative refinement |\n\n## Interactive segmentation\n\nDetails, code examples and parameter tables: [references\u002Finteractive-segmentation.md](references\u002Finteractive-segmentation.md). Read it when this step applies.\n\n## Automatic mask generation\n\n### Basic automatic segmentation\n\n```python\nfrom segment_anything import SamAutomaticMaskGenerator\n\n# Create generator\nmask_generator = SamAutomaticMaskGenerator(sam)\n\n# Generate all masks\nmasks = mask_generator.generate(image)\n\n# Each mask contains:\n# - segmentation: binary mask\n# - bbox: [x, y, w, h]\n# - area: pixel count\n# - predicted_iou: quality score\n# - stability_score: robustness score\n# - point_coords: generating point\n```\n\n### Customized generation\n\n```python\nmask_generator = SamAutomaticMaskGenerator(\n    model=sam,\n    points_per_side=32,          # Grid density (more = more masks)\n    pred_iou_thresh=0.88,        # Quality threshold\n    stability_score_thresh=0.95,  # Stability threshold\n    crop_n_layers=1,             # Multi-scale crops\n    crop_n_points_downscale_factor=2,\n    min_mask_region_area=100,    # Remove tiny masks\n)\n\nmasks = mask_generator.generate(image)\n```\n\n### Filtering masks\n\n```python\n# Sort by area (largest first)\nmasks = sorted(masks, key=lambda x: x['area'], reverse=True)\n\n# Filter by predicted IoU\nhigh_quality = [m for m in masks if m['predicted_iou'] > 0.9]\n\n# Filter by stability score\nstable_masks = [m for m in masks if m['stability_score'] > 0.95]\n```\n\n## Batched inference\n\n### Multiple images\n\n```python\n# Process multiple images efficiently\nimages = [cv2.imread(f\"image_{i}.jpg\") for i in range(10)]\n\nall_masks = []\nfor image in images:\n    predictor.set_image(image)\n    masks, _, _ = predictor.predict(\n        point_coords=np.array([[500, 375]]),\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n    all_masks.append(masks)\n```\n\n### Multiple prompts per image\n\n```python\n# Process multiple prompts efficiently (one image encoding)\npredictor.set_image(image)\n\n# Batch of point prompts\npoints = [\n    np.array([[100, 100]]),\n    np.array([[200, 200]]),\n    np.array([[300, 300]])\n]\n\nall_masks = []\nfor point in points:\n    masks, scores, _ = predictor.predict(\n        point_coords=point,\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n    all_masks.append(masks[np.argmax(scores)])\n```\n\n## ONNX deployment\n\n### Export model\n\n```bash\npython scripts\u002Fexport_onnx_model.py \\\n    --checkpoint sam_vit_h_4b8939.pth \\\n    --model-type vit_h \\\n    --output sam_onnx.onnx \\\n    --return-single-mask\n```\n\n### Use ONNX model\n\n```python\nimport onnxruntime\n\n# Load ONNX model\nort_session = onnxruntime.InferenceSession(\"sam_onnx.onnx\")\n\n# Run inference (image embeddings computed separately)\nmasks = ort_session.run(\n    None,\n    {\n        \"image_embeddings\": image_embeddings,\n        \"point_coords\": point_coords,\n        \"point_labels\": point_labels,\n        \"mask_input\": np.zeros((1, 1, 256, 256), dtype=np.float32),\n        \"has_mask_input\": np.array([0], dtype=np.float32),\n        \"orig_im_size\": np.array([h, w], dtype=np.float32)\n    }\n)\n```\n\n## Common workflows\n\n### Workflow 1: Annotation tool\n\n```python\nimport cv2\n\n# Load model\npredictor = SamPredictor(sam)\npredictor.set_image(image)\n\ndef on_click(event, x, y, flags, param):\n    if event == cv2.EVENT_LBUTTONDOWN:\n        # Foreground point\n        masks, scores, _ = predictor.predict(\n            point_coords=np.array([[x, y]]),\n            point_labels=np.array([1]),\n            multimask_output=True\n        )\n        # Display best mask\n        display_mask(masks[np.argmax(scores)])\n```\n\n### Workflow 2: Object extraction\n\n```python\ndef extract_object(image, point):\n    \"\"\"Extract object at point with transparent background.\"\"\"\n    predictor.set_image(image)\n\n    masks, scores, _ = predictor.predict(\n        point_coords=np.array([point]),\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n\n    best_mask = masks[np.argmax(scores)]\n\n    # Create RGBA output\n    rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)\n    rgba[:, :, :3] = image\n    rgba[:, :, 3] = best_mask * 255\n\n    return rgba\n```\n\n### Workflow 3: Medical image segmentation\n\n```python\n# Process medical images (grayscale to RGB)\nmedical_image = cv2.imread(\"scan.png\", cv2.IMREAD_GRAYSCALE)\nrgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)\n\npredictor.set_image(rgb_image)\n\n# Segment region of interest\nmasks, scores, _ = predictor.predict(\n    box=np.array([x1, y1, x2, y2]),  # ROI bounding box\n    multimask_output=True\n)\n```\n\n## Output format\n\n### Mask data structure\n\n```python\n# SamAutomaticMaskGenerator output\n{\n    \"segmentation\": np.ndarray,  # H×W binary mask\n    \"bbox\": [x, y, w, h],        # Bounding box\n    \"area\": int,                 # Pixel count\n    \"predicted_iou\": float,      # 0-1 quality score\n    \"stability_score\": float,    # 0-1 robustness score\n    \"crop_box\": [x, y, w, h],    # Generation crop region\n    \"point_coords\": [[x, y]],    # Input point\n}\n```\n\n### COCO RLE format\n\n```python\nfrom pycocotools import mask as mask_utils\n\n# Encode mask to RLE\nrle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))\nrle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\n\n# Decode RLE to mask\ndecoded_mask = mask_utils.decode(rle)\n```\n\n## Performance optimization\n\n### GPU memory\n\n```python\n# Use smaller model for limited VRAM\nsam = sam_model_registry[\"vit_b\"](checkpoint=\"sam_vit_b_01ec64.pth\")\n\n# Process images in batches\n# Clear CUDA cache between large batches\ntorch.cuda.empty_cache()\n```\n\n### Speed optimization\n\n```python\n# Use half precision\nsam = sam.half()\n\n# Reduce points for automatic generation\nmask_generator = SamAutomaticMaskGenerator(\n    model=sam,\n    points_per_side=16,  # Default is 32\n)\n\n# Use ONNX for deployment\n# Export with --return-single-mask for faster inference\n```\n\n## Common issues\n\n| Issue | Solution |\n|-------|----------|\n| Out of memory | Use ViT-B model, reduce image size |\n| Slow inference | Use ViT-B, reduce points_per_side |\n| Poor mask quality | Try different prompts, use box + points |\n| Edge artifacts | Use stability_score filtering |\n| Small objects missed | Increase points_per_side |\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Batching, fine-tuning, integration\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\n\n## Resources\n\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything\n- **Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.02643\n- **Demo**: https:\u002F\u002Fsegment-anything.com\n- **SAM 2 (Video)**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2\n- **HuggingFace**: https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fsam-vit-huge\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `clip`: OpenAI's model connecting vision and language.\n- `generate-image`: Generate or edit images with AI models through the OpenRouter Image API (Gemini, Seedream, Recraft, GPT-Image, Riverflow).\n- `blip-2-vision-language`: Explains how to use Salesforce BLIP-2 (Q-Former bridging a frozen image encoder and an LLM such as OPT or FlanT5) through HuggingFace Trans…\n","# Segment Anything Model (SAM)\n\nComprehensive guide to using Meta AI's Segment Anything Model for zero-shot image segmentation.\n\n## When to use SAM\n\n**Use SAM when:**\n- Need to segment any object in images without task-specific training\n- Building interactive annotation tools with point\u002Fbox prompts\n- Generating training data for other vision models\n- Need zero-shot transfer to new image domains\n- Building object detection\u002Fsegmentation pipelines\n- Processing medical, satellite, or domain-specific images\n\n**Key features:**\n- **Zero-shot segmentation**: Works on any image domain without fine-tuning\n- **Flexible prompts**: Points, bounding boxes, or previous masks\n- **Automatic segmentation**: Generate all object masks automatically\n- **High quality**: Trained on 1.1 billion masks from 11 million images\n- **Multiple model sizes**: ViT-B (fastest), ViT-L, ViT-H (most accurate)\n- **ONNX export**: Deploy in browsers and edge devices\n\n**Use alternatives instead:**\n- **YOLO\u002FDetectron2**: For real-time object detection with classes\n- **Mask2Former**: For semantic\u002Fpanoptic segmentation with categories\n- **GroundingDINO + SAM**: For text-prompted segmentation\n- **SAM 2**: For video segmentation tasks\n\n## Quick start\n\n### Installation\n\n```bash\n# From GitHub\npip install git+https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything.git\n\n# Optional dependencies\npip install opencv-python pycocotools matplotlib\n\n# Or use HuggingFace transformers\npip install transformers\n```\n\n### Download checkpoints\n\n```bash\n# ViT-H (largest, most accurate) - 2.4GB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_h_4b8939.pth\n\n# ViT-L (medium) - 1.2GB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_l_0b3195.pth\n\n# ViT-B (smallest, fastest) - 375MB\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fsegment_anything\u002Fsam_vit_b_01ec64.pth\n```\n\n### Basic usage with SamPredictor\n\n```python\nimport numpy as np\nfrom segment_anything import sam_model_registry, SamPredictor\n\n# Load model\nsam = sam_model_registry[\"vit_h\"](checkpoint=\"sam_vit_h_4b8939.pth\")\nsam.to(device=\"cuda\")\n\n# Create predictor\npredictor = SamPredictor(sam)\n\n# Set image (computes embeddings once)\nimage = cv2.imread(\"image.jpg\")\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\npredictor.set_image(image)\n\n# Predict with point prompts\ninput_point = np.array([[500, 375]])  # (x, y) coordinates\ninput_label = np.array([1])  # 1 = foreground, 0 = background\n\nmasks, scores, logits = predictor.predict(\n    point_coords=input_point,\n    point_labels=input_label,\n    multimask_output=True  # Returns 3 mask options\n)\n\n# Select best mask\nbest_mask = masks[np.argmax(scores)]\n```\n\n### HuggingFace Transformers\n\n```python\nimport torch\nfrom PIL import Image\nfrom transformers import SamModel, SamProcessor\n\n# Load model and processor\nmodel = SamModel.from_pretrained(\"facebook\u002Fsam-vit-huge\")\nprocessor = SamProcessor.from_pretrained(\"facebook\u002Fsam-vit-huge\")\nmodel.to(\"cuda\")\n\n# Process image with point prompt\nimage = Image.open(\"image.jpg\")\ninput_points = [[[450, 600]]]  # Batch of points\n\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\")\ninputs = {k: v.to(\"cuda\") for k, v in inputs.items()}\n\n# Generate masks\nwith torch.no_grad():\n    outputs = model(**inputs)\n\n# Post-process masks to original size\nmasks = processor.image_processor.post_process_masks(\n    outputs.pred_masks.cpu(),\n    inputs[\"original_sizes\"].cpu(),\n    inputs[\"reshaped_input_sizes\"].cpu()\n)\n```\n\n## Core concepts\n\n### Model architecture\n\n```\nSAM Architecture:\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\n│  Image Encoder  │────▶│ Prompt Encoder  │────▶│  Mask Decoder   │\n│     (ViT)       │     │ (Points\u002FBoxes)  │     │ (Transformer)   │\n└─────────────────┘     └─────────────────┘     └─────────────────┘\n        │                       │                       │\n   Image Embeddings      Prompt Embeddings         Masks + IoU\n   (computed once)       (per prompt)             predictions\n```\n\n### Model variants\n\n| Model | Checkpoint | Size | Speed | Accuracy |\n|-------|------------|------|-------|----------|\n| ViT-H | `vit_h` | 2.4 GB | Slowest | Best |\n| ViT-L | `vit_l` | 1.2 GB | Medium | Good |\n| ViT-B | `vit_b` | 375 MB | Fastest | Good |\n\n### Prompt types\n\n| Prompt | Description | Use Case |\n|--------|-------------|----------|\n| Point (foreground) | Click on object | Single object selection |\n| Point (background) | Click outside object | Exclude regions |\n| Bounding box | Rectangle around object | Larger objects |\n| Previous mask | Low-res mask input | Iterative refinement |\n\n## Interactive segmentation\n\nDetails, code examples and parameter tables: [references\u002Finteractive-segmentation.md](references\u002Finteractive-segmentation.md). Read it when this step applies.\n\n## Automatic mask generation\n\n### Basic automatic segmentation\n\n```python\nfrom segment_anything import SamAutomaticMaskGenerator\n\n# Create generator\nmask_generator = SamAutomaticMaskGenerator(sam)\n\n# Generate all masks\nmasks = mask_generator.generate(image)\n\n# Each mask contains:\n# - segmentation: binary mask\n# - bbox: [x, y, w, h]\n# - area: pixel count\n# - predicted_iou: quality score\n# - stability_score: robustness score\n# - point_coords: generating point\n```\n\n### Customized generation\n\n```python\nmask_generator = SamAutomaticMaskGenerator(\n    model=sam,\n    points_per_side=32,          # Grid density (more = more masks)\n    pred_iou_thresh=0.88,        # Quality threshold\n    stability_score_thresh=0.95,  # Stability threshold\n    crop_n_layers=1,             # Multi-scale crops\n    crop_n_points_downscale_factor=2,\n    min_mask_region_area=100,    # Remove tiny masks\n)\n\nmasks = mask_generator.generate(image)\n```\n\n### Filtering masks\n\n```python\n# Sort by area (largest first)\nmasks = sorted(masks, key=lambda x: x['area'], reverse=True)\n\n# Filter by predicted IoU\nhigh_quality = [m for m in masks if m['predicted_iou'] > 0.9]\n\n# Filter by stability score\nstable_masks = [m for m in masks if m['stability_score'] > 0.95]\n```\n\n## Batched inference\n\n### Multiple images\n\n```python\n# Process multiple images efficiently\nimages = [cv2.imread(f\"image_{i}.jpg\") for i in range(10)]\n\nall_masks = []\nfor image in images:\n    predictor.set_image(image)\n    masks, _, _ = predictor.predict(\n        point_coords=np.array([[500, 375]]),\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n    all_masks.append(masks)\n```\n\n### Multiple prompts per image\n\n```python\n# Process multiple prompts efficiently (one image encoding)\npredictor.set_image(image)\n\n# Batch of point prompts\npoints = [\n    np.array([[100, 100]]),\n    np.array([[200, 200]]),\n    np.array([[300, 300]])\n]\n\nall_masks = []\nfor point in points:\n    masks, scores, _ = predictor.predict(\n        point_coords=point,\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n    all_masks.append(masks[np.argmax(scores)])\n```\n\n## ONNX deployment\n\n### Export model\n\n```bash\npython scripts\u002Fexport_onnx_model.py \\\n    --checkpoint sam_vit_h_4b8939.pth \\\n    --model-type vit_h \\\n    --output sam_onnx.onnx \\\n    --return-single-mask\n```\n\n### Use ONNX model\n\n```python\nimport onnxruntime\n\n# Load ONNX model\nort_session = onnxruntime.InferenceSession(\"sam_onnx.onnx\")\n\n# Run inference (image embeddings computed separately)\nmasks = ort_session.run(\n    None,\n    {\n        \"image_embeddings\": image_embeddings,\n        \"point_coords\": point_coords,\n        \"point_labels\": point_labels,\n        \"mask_input\": np.zeros((1, 1, 256, 256), dtype=np.float32),\n        \"has_mask_input\": np.array([0], dtype=np.float32),\n        \"orig_im_size\": np.array([h, w], dtype=np.float32)\n    }\n)\n```\n\n## Common workflows\n\n### Workflow 1: Annotation tool\n\n```python\nimport cv2\n\n# Load model\npredictor = SamPredictor(sam)\npredictor.set_image(image)\n\ndef on_click(event, x, y, flags, param):\n    if event == cv2.EVENT_LBUTTONDOWN:\n        # Foreground point\n        masks, scores, _ = predictor.predict(\n            point_coords=np.array([[x, y]]),\n            point_labels=np.array([1]),\n            multimask_output=True\n        )\n        # Display best mask\n        display_mask(masks[np.argmax(scores)])\n```\n\n### Workflow 2: Object extraction\n\n```python\ndef extract_object(image, point):\n    \"\"\"Extract object at point with transparent background.\"\"\"\n    predictor.set_image(image)\n\n    masks, scores, _ = predictor.predict(\n        point_coords=np.array([point]),\n        point_labels=np.array([1]),\n        multimask_output=True\n    )\n\n    best_mask = masks[np.argmax(scores)]\n\n    # Create RGBA output\n    rgba = np.zeros((image.shape[0], image.shape[1], 4), dtype=np.uint8)\n    rgba[:, :, :3] = image\n    rgba[:, :, 3] = best_mask * 255\n\n    return rgba\n```\n\n### Workflow 3: Medical image segmentation\n\n```python\n# Process medical images (grayscale to RGB)\nmedical_image = cv2.imread(\"scan.png\", cv2.IMREAD_GRAYSCALE)\nrgb_image = cv2.cvtColor(medical_image, cv2.COLOR_GRAY2RGB)\n\npredictor.set_image(rgb_image)\n\n# Segment region of interest\nmasks, scores, _ = predictor.predict(\n    box=np.array([x1, y1, x2, y2]),  # ROI bounding box\n    multimask_output=True\n)\n```\n\n## Output format\n\n### Mask data structure\n\n```python\n# SamAutomaticMaskGenerator output\n{\n    \"segmentation\": np.ndarray,  # H×W binary mask\n    \"bbox\": [x, y, w, h],        # Bounding box\n    \"area\": int,                 # Pixel count\n    \"predicted_iou\": float,      # 0-1 quality score\n    \"stability_score\": float,    # 0-1 robustness score\n    \"crop_box\": [x, y, w, h],    # Generation crop region\n    \"point_coords\": [[x, y]],    # Input point\n}\n```\n\n### COCO RLE format\n\n```python\nfrom pycocotools import mask as mask_utils\n\n# Encode mask to RLE\nrle = mask_utils.encode(np.asfortranarray(mask.astype(np.uint8)))\nrle[\"counts\"] = rle[\"counts\"].decode(\"utf-8\")\n\n# Decode RLE to mask\ndecoded_mask = mask_utils.decode(rle)\n```\n\n## Performance optimization\n\n### GPU memory\n\n```python\n# Use smaller model for limited VRAM\nsam = sam_model_registry[\"vit_b\"](checkpoint=\"sam_vit_b_01ec64.pth\")\n\n# Process images in batches\n# Clear CUDA cache between large batches\ntorch.cuda.empty_cache()\n```\n\n### Speed optimization\n\n```python\n# Use half precision\nsam = sam.half()\n\n# Reduce points for automatic generation\nmask_generator = SamAutomaticMaskGenerator(\n    model=sam,\n    points_per_side=16,  # Default is 32\n)\n\n# Use ONNX for deployment\n# Export with --return-single-mask for faster inference\n```\n\n## Common issues\n\n| Issue | Solution |\n|-------|----------|\n| Out of memory | Use ViT-B model, reduce image size |\n| Slow inference | Use ViT-B, reduce points_per_side |\n| Poor mask quality | Try different prompts, use box + points |\n| Edge artifacts | Use stability_score filtering |\n| Small objects missed | Increase points_per_side |\n\n## References\n\n- **[Advanced Usage](references\u002Fadvanced-usage.md)** - Batching, fine-tuning, integration\n- **[Troubleshooting](references\u002Ftroubleshooting.md)** - Common issues and solutions\n\n## Resources\n\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything\n- **Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.02643\n- **Demo**: https:\u002F\u002Fsegment-anything.com\n- **SAM 2 (Video)**: https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2\n- **HuggingFace**: https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fsam-vit-huge\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `clip`: OpenAI's model connecting vision and language.\n- `generate-image`: Generate or edit images with AI models through the OpenRouter Image API (Gemini, Seedream, Recraft, GPT-Image, Riverflow).\n- `blip-2-vision-language`: Explains how to use Salesforce BLIP-2 (Q-Former bridging a frozen image encoder and an LLM such as OPT or FlanT5) through HuggingFace Trans…\n",{"name":5,"license":15,"metadata":53,"description":7},{"tags":54,"version":55,"category":56,"maintainer":57,"dependencies":58},"Multimodal, Image Segmentation, Computer Vision, SAM, Zero-Shot","1.0.0","multimodal-and-emerging","Kalaris Labs","segment-anything, transformers>=4.30.0, torch>=1.7.0","main","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model",468,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-11 02:22:54",{"source":8,"skillName":5,"commands":70,"prompt":80,"zipAvailable":81},[71,74,77],{"tool":72,"command":73},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill segment-anything-model",{"tool":75,"command":76},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill segment-anything-model",{"tool":78,"command":79},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill segment-anything-model","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002FSKILL.md",true,[83,89,93],{"path":84,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":87,"sourceHtmlUrl":88},"references\u002Fadvanced-usage.md","REFERENCE","参考文档",15803,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002Freferences\u002Fadvanced-usage.md",{"path":90,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":91,"sourceHtmlUrl":92},"references\u002Finteractive-segmentation.md",1598,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002Freferences\u002Finteractive-segmentation.md",{"path":94,"resourceType":85,"resourceTypeDesc":86,"sizeBytes":95,"sourceHtmlUrl":96},"references\u002Ftroubleshooting.md",11914,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fsegment-anything-model\u002Freferences\u002Ftroubleshooting.md",[98,124,149,176,195,209],{"skillNo":99,"name":100,"nameZh":6,"description":101,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":102,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":48},"01M716A6ETNB9B1SQ3GXZC75VR","deja-search","deja-vu memory — search the user's past AI coding sessions with the deja CLI. Holds what they already settled and never wrote into the repo (naming rules, banned or already-tried tools, required checks before merge, preferred values, who owns what). Use before your first edit in a task, before you call a change done or ready to merge, before you recommend a tool, library, design or reviewer, before re-debugging an error that may already be solved, when they say 'didn't we fix this before', 'what did we decide about X' or 'I already have that', and before telling them something on this machine does not exist.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-search\u002FSKILL.md",[],[106,107,108,109,110,111,112,113,114,115,116,32,35,117,118,119,120,121,122,123],"search","deja search","error","ready","never","memory","re","decide","fix","design","debugging","done","reviewer","io","change","cli","merge","past",{"skillNo":125,"name":126,"nameZh":6,"description":127,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":102,"licenseFileUrl":6,"sourceHtmlUrl":128,"categories":129,"keywords":130,"publishedAt":48},"01M716A3J2W5SENSE9VGMXB375","deja-reflect","End-of-session reflection and memory curation with deja-vu. Review what was decided, what commands broke and what repaired them, and record durable project rules or decisions with deja remember so the next session starts with them.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-reflect\u002FSKILL.md",[],[131,132,133,134,111,135,136,137,138,139,140,141,142,119,143,144,145,146,147,148],"reflect","deja reflect","brew","remember","project rules","project","session","end","reflection","Rules","decisions","review","with","record","github","next","https","node",{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":102,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":160,"publishedAt":48},"01M715T3RR4BAYSR57HPPTJ0PF","revid","Make, edit, export and post AI videos with Revid (revid.ai) through the Revid MCP server. Use whenever the user wants a short-form or long-form video made (from an idea, a script, a URL, an article, a PDF, a tweet, a product page, their own clips, or a song), wants to find what is going viral on TikTok, YouTube, Instagram or X before making something, wants to edit or re-export an existing Revid project, generate a standalone AI image or clip, make a series episode, or publish and schedule a finished video to connected social accounts. Covers setup (hosted MCP with OAuth, or an API key), the 36 tools, credits, polling and the rules for publishing. Routes to the more specific revid-* skills when one fits.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftmakersas\u002Frevid-skills\u002Fskills\u002Frevid\u002FSKILL.md",[155,158,159],{"categoryNo":156,"categoryName":157,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},[151,161,162,163,164,165,136,166,112,167,168,169,170,32,171,39,172,173,174,175],"Video","short-form","standalone","PDF","viral","tiktok","short","social","MCP","song","URL","mcp server","schedule","api","publish",{"skillNo":177,"name":178,"nameZh":6,"description":179,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":102,"licenseFileUrl":6,"sourceHtmlUrl":180,"categories":181,"keywords":182,"publishedAt":48},"01M71569ZM2X3H5RBFK8PVKMK2","review-study","Review a QuestionPunk survey before launch for wording, answer choices, branching, and missing settings. Use for check my survey or review this draft before publishing.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fquestionpunk\u002Fcodex-plugin\u002Fskills\u002Freview-study\u002FSKILL.md",[],[183,184,142,33,185,186,187,188,41,189,190,191,192,193,194],"study","review study","answer","missing","survey","publishing","check","draft","settings","launch","my","branching",{"skillNo":196,"name":197,"nameZh":6,"description":198,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":102,"licenseFileUrl":6,"sourceHtmlUrl":199,"categories":200,"keywords":201,"publishedAt":48},"01M71567GV0C35YQBY0DPGR86B","manage-study","Find QuestionPunk studies, check status, publish an explicitly requested draft, or retrieve its participant link. Use for find my survey, publish this study, or how many people responded.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fquestionpunk\u002Fcodex-plugin\u002Fskills\u002Fmanage-study\u002FSKILL.md",[],[202,183,203,33,187,41,204,189,190,175,205,206,207,208,193],"manage study","manage","people","how","find","link","status",{"skillNo":210,"name":211,"nameZh":6,"description":212,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":102,"licenseFileUrl":6,"sourceHtmlUrl":213,"categories":214,"keywords":215,"publishedAt":48},"01M71565A8TCA01AZQZRGRYMFZ","create-study","Create a saved QuestionPunk survey draft from a research brief or edit specified questions. Use for requests such as make a customer feedback survey or shorten my onboarding study.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fquestionpunk\u002Fcodex-plugin\u002Fskills\u002Fcreate-study\u002FSKILL.md",[],[216,183,217,218,38,219,33,220,187,221,41,222,223,190,224,225,226,227,228,193],"create study","create","brief","feedback","customer feedback","research brief","shorten","saved","research","requests","customer","questions","edit"]