AGENT SKILL

multimodal-video

Native video understanding, event timestamping, frame sampling, and video narrative generation.

License 许可
MIT
最近更新
2026-06-05
Stars
★ 2
Forks
0

01 前言 Front Matter

name
multimodal-video
license
MIT license
metadata
{ "skill-author": "Lord1Egypt" }
description
Native video understanding, event timestamping, frame sampling, and video narrative generation.
allowed-tools
Read Write Edit Bash

03 SKILL.md 正文

Multimodal Video Intelligence Skill

Overview

This skill outlines how to leverage Google's Gemini models (such as gemini-2.5-flash or gemini-2.5-pro) to process video files natively without pre-extracting audio or individual image frames manually.

Gemini accepts direct video file inputs (up to 50MB directly in generation, or up to 2GB via the File API) and performs multimodal fusion over audio tracks, spatial layouts, and temporal transitions.


When to Use This Skill

  • For temporal query-answering (e.g., "At what timestamp does the red car appear?").
  • Summarizing lecture, security camera, or gaming videos.
  • Extracting specific textual details or visual overlays from video sequences.
  • Translating or transcribing speech combined with on-screen visual context.

Quick Start (with runnable code examples)

import time
from google import genai

# Initialize the Gemini GenAI Client
client = genai.Client()

def analyze_video(video_path: str):
    print("Uploading video file to Gemini File API...")
    video_file = client.files.upload(file=video_path)
    
    # Poll for file state until it transitions from PROCESSING to ACTIVE
    while video_file.state.name == "PROCESSING":
        print("Waiting for video to be processed by Google servers...")
        time.sleep(5)
        video_file = client.files.get(name=video_file.name)
        
    if video_file.state.name != "ACTIVE":
        raise ValueError(f"File upload failed with state: {video_file.state.name}")
        
    print(f"Video file is ready! URI: {video_file.uri}")
    
    # Query the video using gemini-2.5-flash
    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            video_file,
            "Provide a bulleted timeline of events in this video. Format each event as [MM:SS] - Description."
        ]
    )
    
    print("\n--- Video Timeline Analysis ---")
    print(response.text)
    
    # Cleanup: Delete file from Gemini storage after processing
    print("\nCleaning up remote video file...")
    client.files.delete(name=video_file.name)

if __name__ == "__main__":
    # Example video path
    analyze_video("sample_video.mp4")

Advanced Usage

Temporal Search (Visual Event Identification)

To locate highly specific occurrences inside a long clip, specify the frame rate or use clear timeline descriptions in the prompt:

response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        video_file,
        "Examine the video closely. Find the exact start and end timestamps where a person is wearing a blue shirt. Output ONLY the timestamps in JSON format: {'start': 'MM:SS', 'end': 'MM:SS'}."
    ],
    config=dict(
        response_mime_type="application/json"
    )
)

Key References


Dependencies

  • google-genai>=0.1.1
  • ffmpeg (optional, for local video clipping/resizing before upload)

04 同仓库更多 Skills

Lord1Egypt
Lord1Egypt/ai-skillforge

zarr-python

Chunked N-D arrays for cloud storage. Compressed arrays, parallel I/O, S3/GCS integration, NumPy/Dask/Xarray compatible, for large-scale scientific computing pipelines.

★ 2
Lord1Egypt
Lord1Egypt/ai-skillforge

what-if-oracle

Run structured What-If scenario analysis with multi-branch possibility exploration. Use this skill when the user asks speculative questions like "what if...", "what would happen if...", "what are the possibilities", "explore scenarios", "scenario analysis", "possibility space", "what could go wrong", "best case / worst case", "risk analysis", "contingency planning", "strategic options", or any question about uncertain futures. Also trigger when the user faces a fork-in-the-road decision, wants to stress-test an idea, or needs to think through consequences before committing.

★ 2
Lord1Egypt
Lord1Egypt/ai-skillforge

venue-templates

Access comprehensive LaTeX templates, formatting requirements, and submission guidelines for major scientific publication venues (Nature, Science, PLOS, IEEE, ACM), academic conferences (NeurIPS, ICML, CVPR, CHI), research posters, and grant proposals (NSF, NIH, DOE, DARPA). This skill should be used when preparing manuscripts for journal submission, conference papers, research posters, or grant proposals and need venue-specific formatting requirements and templates.

★ 2
Lord1Egypt
Lord1Egypt/ai-skillforge

vaex

Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV/HDF5/Arrow/Parquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that do not fit in memory.

★ 2
Lord1Egypt
Lord1Egypt/ai-skillforge

usfiscaldata

Query the U.S. Treasury Fiscal Data API for federal financial data including national debt, government spending, revenue, interest rates, exchange rates, and savings bonds. Access 54 datasets and 182 data tables with no API key required. Use when working with U.S. federal fiscal data, national debt tracking (Debt to the Penny), Daily Treasury Statements, Monthly Treasury Statements, Treasury securities auctions, interest rates on Treasury securities, foreign exchange rates, savings bonds, or any U.S. government financial statistics.

★ 2
Lord1Egypt
Lord1Egypt/ai-skillforge

treatment-plans

Generate concise (3-4 page), focused medical treatment plans in LaTeX/PDF format for all clinical specialties. Supports general medical treatment, rehabilitation therapy, mental health care, chronic disease management, perioperative care, and pain management. Includes SMART goal frameworks, evidence-based interventions with minimal text citations, regulatory compliance (HIPAA), and professional formatting. Prioritizes brevity and clinical actionability.

★ 2