social-video-mining
Extracts insights from short video (TikTok, Shorts, Reels, Douyin) without paid speech-to-text: yt-dlp, ffmpeg and vision over the frames. Use it when on-screen text carries the signal or when there is no transcript.
- License 许可
- MIT
- 最近更新
- 2026-11-14
- Stars
- ★ 18
- Forks
- 6
01 前言 Front Matter
- name
- social-video-mining
- description
- Extracts insights from short video (TikTok, Shorts, Reels, Douyin) without paid speech-to-text: yt-dlp, ffmpeg and vision over the frames. Use it when on-screen text carries the signal or when there is no transcript.
02 资源文件 1
- 参考文档skill.json334 B
03 SKILL.md 正文
Social Video Mining — Frames Over Audio
Technique skill, not a tool wrapper. Documents the workflow we used to mine OSS-replacement gems from a daily-tech-tip TikTok creator without burning OpenAI Whisper credits. The key insight: tech short-form video puts the signal on screen, not in the audio — project names, URLs, GitHub stats, license badges all appear as overlay text and screen recordings.
When to use
- Curating skill candidates from a tech-creator channel (TikTok, YouTube Shorts, Reels, Douyin, Bilibili)
- Verifying product claims from video reviews without watching dozens of videos
- Building reference material from conference talks where transcripts are unavailable
- Any "video → structured knowledge" task where audio transcription is blocked, expensive, or just not the highest-signal channel
When NOT to use
- Audio IS the signal (interviews, podcasts, lectures) — use
transcribeskill instead - Long-form video where the project name only appears once — frame sampling will miss it; consider audio transcription
- Live streams or videos behind paywalls
Workflow
1. List → yt-dlp --flat-playlist --dump-json --playlist-end N <channel-url>
Extract: title, description, view_count, upload_date per video
2. Triage → filter titles/descriptions for the signal you want
Sort by view_count (popularity = social validation)
3. Download → yt-dlp -q -o "/tmp/<prefix>_%(id)s.%(ext)s" <video-url>
4. Frames → ffmpeg -ss <sec> -i <video> -frames:v 1 -q:v 4 <frame>.jpg
Extract 2-3 frames per video at strategic timestamps:
- t=2-5s : intro / project name often appears
- t=15-25s : middle / detailed view of the tool
- t=35-40s : outro / CTA + URL often appears
5. Vision → Read each JPG — native multimodal sees:
- Project names (top-left logos, browser tabs)
- GitHub repo URLs (README banners, browser URL bar)
- License badges (MIT / Apache / AGPL)
- Star counts, version numbers
- On-screen overlay text (Spanish captions, hashtags)
6. Verify → WebFetch or `gh search repos "<name>"` to confirm canonical URL exists
and license / stars match what was on screen
7. Cleanup → rm -rf /tmp/<prefix>_* to release ~5MB per video
Strategic timestamp picks
Most tech-tip short-form videos follow this structure:
| Timestamp | Typical content |
|---|---|
| 0-3s | Hook + caption with project pitch ("AWS en local y gratis") |
| 3-10s | Project logo + landing page hero ("Local AWS. Zero cost.") |
| 10-25s | Demo / detail screen recording with on-screen features |
| 25-35s | Stats / license / GitHub repo screenshot |
| 35-45s | Outro CTA with URL + hashtags |
Sample frames at t=5, t=20, t=40 and you'll capture name, demo, and source URL in 90% of cases. Add t=12, t=32 for hard cases.
Cost profile (vs alternatives)
| Approach | Tokens / 1 video | API cost / 1 video | Speed |
|---|---|---|---|
| WebFetch on TikTok URL | ~500 (bot-walled) | $0 | Fast, useless |
| Browser scrape via agent-browser | ~5,000-10,000 | $0 | Slow, login-walled |
| yt-dlp + OpenAI Whisper transcription | ~3,000 | ~$0.01 | Medium, needs API key |
| This skill (yt-dlp + frames + native vision) | ~2,000 (3 frames × ~600 each) | $0 | Fast, no API key |
Native vision dominates for project-identification tasks. Transcription only wins when audio is the signal (which short-form tech video rarely is).
Permissions / install prerequisites
~/.local/bin/yt-dlp— install via standalone GitHub release (no PEP 668 conflict). Settings allowlist needs:Bash(curl -fsSL https://github.com/yt-dlp/yt-dlp/releases/*)Bash(chmod +x /home/<user>/.local/bin/yt-dlp)Bash(~/.local/bin/yt-dlp:*)
ffmpeg— already present on most systems at~/.local/bin/ffmpegor viawhich ffmpegjq— for parsing yt-dlp's JSON dumps- No external API keys required
Cleanup discipline
Always rm -rf /tmp/<prefix>_* at the end of a mining run. Each video is ~5MB; a 10-video session = 50MB. Don't leave artifacts.
Limits / when this fails
- Channel doesn't permit unauthenticated metadata access — yt-dlp may need cookies; falls back to agent-browser
- Creator uses CapCut / Premiere with minimal on-screen text — name only spoken, never shown. Then audio transcription wins
- Branded text obscured by stylized fonts / shaders — vision misreads; use 2 frames to cross-check
- Project name is generic ("API client") — vision identifies the screenshot but not the brand; WebFetch search step is essential
TikTok gotcha: do NOT add curl_cffi, and don't blindly upgrade yt-dlp
Verified 2026-06-11 against a live TikTok pull. Two traps, both counter-intuitive:
- curl_cffi impersonation BREAKS the TikTok extractor. yt-dlp warns "no impersonation target available" on TikTok, which looks like the thing to fix. It is not. Inject
curl_cffiand TikTok serves a challenge page with no rehydration JSON, so extraction dies withERROR: Unable to extract universal data for rehydration. Plain urllib (no curl_cffi) gets the normal page and works. Leave that warning alone for TikTok. - yt-dlp versions after 2026.03.17 regressed TikTok with the same rehydration error (2026.6.9 and master both fail, even without curl_cffi). Pin to a known-working build (
pipx runpip yt-dlp install "yt-dlp==2026.3.17") and only upgrade once an upstream TikTok fix lands. Test the actual target video after any yt-dlp change, never assume newer is safer.
Recognition signal: Unable to extract universal data for rehydration on a TikTok URL = one of these two, not a network blip.
Anti-fabrication discipline (CRITICAL per CLAUDE.md)
Vision can hallucinate text it didn't actually see. Every project name extracted by vision MUST be verified via WebFetch or gh search before writing it into a brain skill or a client recommendation. The verification step is not optional. Workflow step 6 exists specifically to enforce this.
If verification fails (no repo found, name doesn't match search results), do NOT invent a URL or guess. Report "vision saw X but no canonical repo found — manual verification needed" and stop.
Related brain assets
transcribe— when audio IS the signal (interviews, lectures, podcasts)image-analyzer,gemini-vision— for richer per-image analysis when frames need deeper interpretationagent-browser— fallback when yt-dlp is blockedknowledge-corpus— to organize what's mined into a queryable brain layer- Trend Researcher agent — natural consumer of mined social-video gems
- Sister technique: the broader "mine creator channels for OSS gems" pattern that produced this skill,
claude-mem-persistent-memory,bruno-postman-alternative,coolify-self-hosted-paas,spacetimedb-backend-as-database, andvibevoice-elevenlabs-alternativein a single research session
04 同仓库更多 Skills
yeet
Use only when the user explicitly asks to stage, commit, push, and open a GitHub pull request in one flow using the GitHub CLI (`gh`).
wrangler
Cloudflare Workers CLI for deploying, developing, and managing Workers, KV, R2, D1, Vectorize, Hyperdrive, Workers AI, Containers, Queues, Workflows, Pipelines, and Secrets Store. Load before running wrangler commands to ensure correct syntax and best practices. Biases towards retrieval from Cloudflare docs over pre-trained knowledge.
workspace-skill-discovery
Discover and include ALL skills across the workspace — not just global ones. Use at the start of every session or when the user asks about available skills. Ensures project-level skills (under .claude/skills/ or skills/ in any repo) are never missed.
workers-best-practices
Reviews and authors Cloudflare Workers code against production best practices. Load when writing new Workers, reviewing Worker code, configuring wrangler.jsonc, or checking for common Workers anti-patterns (streaming, floating promises, global state, secrets, bindings, observability). Biases towards retrieval from Cloudflare docs over pre-trained knowledge.
winui-app
Create, build and design WinUI 3 desktop apps with C# and the Windows App SDK following Microsoft's official guidance and CommunityToolkit. Covers XAML, navigation, theming, accessibility and deployment.
windows-brain-script-portability
Scripts in ~/.claude/scripts/ that run on a Windows brain: use the python3.cmd shim and not bare python3, and reconfigure stdout to UTF-8 if you print symbols. Use it when writing a brain script or triaging a Windows-only crash.