Inspect video content locally by extracting frames with ffmpeg and transcribing speech with Whisper. The skill produces timestamped image paths, transcript segments, and video details in JSON, giving an agent material for visual review without requiring API keys.
Understand video content locally using ffmpeg frame extraction and Whisper transcription. No API keys needed. Use when: (1) Understanding what a video contains, (2) Transcribing video audio locally, (3) Extracting key frames for visual analysis, (4) Getting video content without API keys.
Review the instructions the agent will receive before you install.
Understand video content locally using ffmpeg for frame extraction and Whisper for transcription. Fully offline, no API keys required.
ffmpeg + ffprobe (required): brew install ffmpegopenai-whisper (optional, for transcription): pip install openai-whisper# Scene detection + transcribe (default)
python3 skills/video-understand/scripts/understand_video.py video.mp4
# Keyframe extraction
python3 skills/video-understand/scripts/understand_video.py video.mp4 -m keyframe
# Regular interval extraction
python3 skills/video-understand/scripts/understand_video.py video.mp4 -m interval
# Limit frames extracted
python3 skills/video-understand/scripts/understand_video.py video.mp4 --max-frames 10
# Use a larger Whisper model
python3 skills/video-understand/scripts/understand_video.py video.mp4 --whisper-model small
# Frames only, skip transcription
python3 skills/video-understand/scripts/understand_video.py video.mp4 --no-transcribe
# Quiet mode (JSON only, no progress)
python3 skills/video-understand/scripts/understand_video.py video.mp4 -q
# Output to file
python3 skills/video-understand/scripts/understand_video.py video.mp4 -o result.json
| Flag | Description |
|---|---|
video | Input video file (positional, required) |
-m, --mode | Extraction mode: scene (default), keyframe, interval |
--max-frames | Maximum frames to keep (default: 20) |
--whisper-model | Whisper model size: tiny, base, small, medium, large (default: base) |
--no-transcribe | Skip audio transcription, extract frames only |
-o, --output | Write result JSON to file instead of stdout |
-q, --quiet | Suppress progress messages, output only JSON |
| Mode | How it works | Best for |
|---|---|---|
scene | Detects scene changes via ffmpeg select='gt(scene,0.3)' | Most videos, varied content |
keyframe | Extracts I-frames (codec keyframes) | Encoded video with natural keyframe placement |
interval | Evenly spaced frames based on duration and max-frames | Fixed sampling, predictable output |
If scene mode detects no scene changes, it automatically falls back to interval mode.
The script outputs JSON to stdout (or file with -o). See references/output-format.md for the full schema.
{
"video": "video.mp4",
"duration": 18.076,
"resolution": {"width": 1224, "height": 1080},
"mode": "scene",
"frames": [
{"path": "/abs/path/frame_0001.jpg", "timestamp": 0.0, "timestamp_formatted": "00:00"}
],
"frame_count": 12,
"transcript": [
{"start": 0.0, "end": 2.5, "text": "Hello and welcome..."}
],
"text": "Full transcript...",
"note": "Use the Read tool to view frame images for visual understanding."
}
Use the Read tool on frame image paths to visually inspect extracted frames.
references/output-format.md -- Full JSON output schema documentationNo reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Create 3D objects from text or images, choose catalog assets for repeated scenery, and inspect the results in Blender. The workflow checks geometry, materials, scale, and ground contact while recording asset sources and generation costs.
Generate background music and songs with lyrics using ACE-Step 1.5, with controls for duration, tempo, and musical key. You can also restyle reference audio and separate vocals or individual instruments from mixed tracks.
Build voice assistants and conversational characters with ElevenLabs. Configure voices, language models, and external tools, then connect conversations to a website widget or use Twilio integration for outbound phone calls.
Generate video clips from text descriptions or reference images through multiple API services. Compare available providers by cost and task needs, choose aspect ratios, track generation progress, and refine existing clips through conversational editing with Gemini Omni.
Create AI video clips from text or reference images, choosing among providers to match the task and budget. Supports vertical social videos, product photo animation, and conversational editing of existing clips with Gemini Omni Flash.
Generate and edit images and videos through Atlas Cloud using one API key across multiple model families. Choose supported models for each task, download the results, and retain details about the model, inputs, and estimated cost.