AGENT SKILL

single-modality

Operate InternVideo2 single-modality visual workflows for pretraining, finetuning, probing, distillation, model selection, datasets, launch adaptation, and troubleshooting.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
single-modality
license
Apache 2.0
metadata
{ "disco-role": "operating", "source-branch": "InternVideo2-single_modality", "source-commit": "3965eef16e2dadd0ea6c8d0cc29c8a3039df52e3" }
description
Operate InternVideo2 single-modality visual workflows for pretraining, finetuning, probing, distillation, model selection, datasets, launch adaptation, and troubleshooting.
disable-model-invocation
true

02 资源文件 4

03 SKILL.md 正文

InternVideo2 Single-Modality Operating Skill

Use this sub-skill when a user asks about the visual-only single_modality branch of InternVideo2: masked video pretraining, action-classification finetuning, linear or attentive probing, distillation from larger teachers, model-zoo choice, dataset/weight paths, SLURM or DeepSpeed launch adaptation, or single-modality failure diagnosis.

Do not use this sub-skill for video-text/audio retrieval, chat/video-MLLM work, InternVideo3, InternVideo-Next, or InternVid dataset acquisition beyond the action-recognition annotation layouts needed here. Route those to the appropriate sibling sub-skill or root repo guidance.

Operating protocol

  1. Classify the requested workflow. Choose one of: pretraining, distillation, full finetuning, linear probing, attentive probing, or eval-only classification. Use references/workflows.md for entry points and command shapes.
  2. Collect required user inputs before executing anything expensive. Confirm dataset split root or pretraining CSV, checkpoint/model root, desired model size, launcher (srun, torchrun, or plain python), GPU count, precision, and whether the user is asking for command construction only or an actual run.
  3. Check prerequisites. Use references/configuration.md to verify Python/CUDA dependencies, INTERNVIDEO2_DATA_PATH, INTERNVIDEO2_MODEL_PATH, annotation CSV layout, teacher weight locations, and model registry names. Full training/evaluation requires user-provided datasets, checkpoints, CUDA-capable dependencies, and cluster resources.
  4. Generate commands safely. Prefer the bundled helper scripts/build_single_modality_command.py to render a dry-run command. The helper prints command text only; it has no training side effects. Review paths, GPUs, batch size, DeepSpeed flags, and dataset class before running.
  5. Adapt launchers deliberately. Source workflows are SLURM-first and usually use DeepSpeed with BF16. For local torchrun, preserve distributed environment variables and port selection. For single-process debugging, remove --enable_deepspeed, --dist_eval, and large batch sizes unless the user explicitly prepared that stack.
  6. Diagnose by symptom. Use references/troubleshooting.md for missing CUDA extensions, teacher checkpoint failures, dataset parsing errors, stale source shell scripts, class-head mismatches, distributed hangs, and OOM recovery.

Quick route table

User intent Use Key inputs
Train Stage1 1B/6B visual model on K-Mash-style data run_pretraining.py recipe in workflows pretraining CSV, InternVL visual teacher, VideoMAEv2-g teacher, large multi-GPU launch
Distill S/B/L from InternVideo2 Stage2-1B run_distill.py recipe in workflows K-Mash CSV, teacher checkpoint registry patched to portable paths, model size
Full tune on Kinetics/K400/K600/K700/MiT/SSV2/ANet/HACS run_finetuning.py recipe in workflows split root with train.csv, val.csv, test.csv; checkpoint; classes; frames/crops
Linear probe or attentive probe run_linear_probing.py recipe in workflows checkpoint, dataset class, open_clip_projector/open_block_num, frame schedule
Select checkpoint/model family configuration model table target dataset, model scale, stage, available weights
Fix a run failure troubleshooting first error line, command, environment summary, dataset/checkpoint existence

Non-negotiable boundaries

  • Do not submit source shell scripts blindly. They are cluster-specific and some contain stale names or non-portable checkpoint assumptions; distill their options into a reviewed command first.
  • Do not claim CPU validation proves training readiness. The selected verification for this skill is static/helper-script verification; native training and evaluation remain optional heavyweight GPU/SLURM work.
  • Do not expose private machine paths, local environment prefixes, or credentials in generated commands or reports.
  • Do not modify user code to patch checkpoint paths or loader bugs without explicit user approval. If a patch is needed, explain the exact portable change and why.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351