dynamic-egomotion-motion-mask-note
Estimate motion cue fragments and dynamic-foreground candidate fragments across frozen sampled steps and sampled frames, then publish an approved sampled motion-mask note with one binding row per target step/frame.
- License 许可
- MIT
- 最近更新
- 2026-09-30
- Stars
- ★ 1
- Forks
- 0
01 前言 Front Matter
- name
- dynamic-egomotion-motion-mask-note
- description
- Estimate motion cue fragments and dynamic-foreground candidate fragments across frozen sampled steps and sampled frames, then publish an approved sampled motion-mask note with one binding row per target step/frame.
03 SKILL.md 正文
Estimate Motion Cue Fragments And Dynamic-Foreground Candidate Fragments
Use this after workflow/dynamic_egomotion_video_checkpoint.json exists. This stage works on the sampled_video_frame_and_step_family surface: compute the reusable motion evidence and dynamic-foreground fragments once, then publish workflow/dynamic_egomotion_motion_mask_note.json for the binder.
Prerequisite Artifact
workflow/dynamic_egomotion_video_checkpoint.json/root/input.mp4
Treat the checkpoint as the authority for sample_rate_fps, sample_indices, sample_step_keys, video_shape, and target_output_paths. Keep sample_rate_fps at 6. Do not rename step keys or write /root/pred_instructions.json or /root/pred_dyn_masks.npz in this stage.
Current-Stage Output
Write only:
workflow/dynamic_egomotion_motion_mask_note.json
Set:
current_stage:dynamic-egomotion-motion-mask-notenext_required_skill:dynamic-egomotion-motion-mask-binderallowed_outputs:['workflow/dynamic_egomotion_motion_mask_note.json']
Sampled Motion-Mask Note
- Resolve the checkpointed sampled positions back into video frames at 6 fps and read only those frames.
- Build
motion_fragment_catalogfrom the checkpointedsample_step_keys. For each sampled step, track corners withgoodFeaturesToTrackpluscalcOpticalFlowPyrLK, fit a robustestimateAffinePartial2Dtransform, and record label-set evidence. Allow multi-label steps. A compact decision rule is enough:abs(scale - 1)above a small threshold ->Dolly InorDolly Outabs(rotation)above a small threshold ->Roll LeftorRoll Right- dominant horizontal translation ->
Pan LeftorPan Right - dominant vertical translation ->
Tilt UporTilt Down - otherwise
Stay
- Build
mask_fragment_catalogfrom the sampled frames. For each sampled frame after the first, warp the previous gray frame into the current frame, derivevalidfrom a warped all-ones mask, thresholdabs(curr - warped_prev)usingmedian + 3 * MADovervalid, apply open/close morphology, and drop small connected components. Store the fragment in sparse form with enough data for later CSR emission. - Build
approved_sample_binding_tablewith one row pertarget_step_keyandtarget_sample_index. Keeprow_local_motion_fragment_handleandrow_local_mask_fragment_handleas the local handles. Select one adjacent sampled frame whose mask fragment has a different CSR cardinality, retain its row-local owner handle, and set itscopied_from_mask_fragment_handleto that adjacent non-self handle. The approved row is now the sole authority for this choice; record the alternate handles but do not reopen the video to reconsider it in later stages. Other rows may remain local. Keep this note reference-oriented. Do not serialize final instruction intervals or final NPZ arrays here. - Run a compact schema check before stopping:
- top-level keys are exactly
sample_rate_fps,sample_indices,sample_step_keys,video_shape,target_output_paths,motion_fragment_catalog,mask_fragment_catalog,approved_sample_binding_table,current_stage,next_required_skill,allowed_outputs - every motion label is one of
Stay,Dolly In,Dolly Out,Pan Left,Pan Right,Tilt Up,Tilt Down,Roll Left,Roll Right - every binding row contains
target_sample_index,target_step_key,row_local_motion_fragment_handle,copied_from_motion_fragment_handle,row_local_mask_fragment_handle,copied_from_mask_fragment_handle,alternate_fragment_handles,binding_mode - any sparse mask fragment still matches
video_shape
- top-level keys are exactly
Output Contract
Write one JSON object with:
sample_rate_fps:6sample_indices:int[]sample_step_keys:string[]video_shape:[H, W]target_output_paths: carriesinstructionsanddyn_masksmotion_fragment_catalog:motion_fragment_handle -> label-set evidence for sampled stepsmask_fragment_catalog:mask_fragment_handle -> dynamic-foreground fragment for sampled framesapproved_sample_binding_table: rows keyed bytarget_step_keyandtarget_sample_indexcurrent_stage:dynamic-egomotion-motion-mask-notenext_required_skill:dynamic-egomotion-motion-mask-binderallowed_outputs:['workflow/dynamic_egomotion_motion_mask_note.json']
Each binding row must contain:
target_sample_indextarget_step_keyrow_local_motion_fragment_handlecopied_from_motion_fragment_handlerow_local_mask_fragment_handlecopied_from_mask_fragment_handlealternate_fragment_handlesbinding_mode
Keep fragment handles opaque.
Continuation Command
python - <<'PY'
import json
note = json.load(open('workflow/dynamic_egomotion_motion_mask_note.json'))
assert note['current_stage'] == 'dynamic-egomotion-motion-mask-note'
assert note['next_required_skill'] == 'dynamic-egomotion-motion-mask-binder'
print('binding_surface_kind=sampled_video_frame_and_step_family')
print('next_required_skill=dynamic-egomotion-motion-mask-binder')
PY
Stop Condition
Stop when workflow/dynamic_egomotion_motion_mask_note.json exists, the compact schema check passes, and the continuation command prints next_required_skill=dynamic-egomotion-motion-mask-binder. Do not open the binder skill or write the final prediction files in this stage.
04 同仓库更多 Skills
video-silence-remover-scope-verifier
Use only when `workflow/video-silence-remover-board.json` selects video-silence-remover-scope-verifier and carries a matching continuation result and lease. Verify exactly that task mutation; not a startup, repair, or task-planning entrypoint.
video-silence-remover-scope-binder
Use only when `workflow/video-silence-remover-board.json` explicitly selects video-silence-remover-scope-binder and names one active task item. Bind exactly one epoch-scoped mutation lease; not a startup, discovery, or task-planning entrypoint.
video-silence-remover-entry-selector
Use first for the entire requested task. Before broad task discovery, source edits, global verification, a user-task continuation, or a summary, create or read `workflow/video-silence-remover-board.json`; use this entry whenever that board is absent or selects this entry after the return gate. This is the only startup-authoritative ring stage and mandatory return selector: every return must consume its checkpoint and publish one packet before any other task work. Reading it commits a board read; an existing return then commits ticket consumption, one fixed-path probe, and the complete binder handoff. The persistent board forbids cleanup and final response.
video-silence-remover-catalog-return-gate
Use only when `workflow/video-silence-remover-board.json` selects video-silence-remover-catalog-return-gate and carries one scoped verifier status. Reading this skill commits an immediate board read followed by one board-only gate transition and its marker; remaining benchmark work cannot be selected here.
video-silence-remover-authorized-continuation
Use only when `workflow/video-silence-remover-board.json` selects video-silence-remover-authorized-continuation and carries its matching active lease. Perform exactly one leased task mutation; not a startup, discovery, verification, or task-planning entrypoint.
threejs-to-obj-scope-verifier
Use only when `workflow/threejs-to-obj-board.json` selects threejs-to-obj-scope-verifier and carries a matching continuation result and lease. Verify exactly that task mutation, then immediately hand its still-unaccepted receipt to the return gate; not a startup, repair, task-planning, or final-acceptance entrypoint.