dundunhan/dsh-video-lens

README excerpt

dsh-video-lens Video understanding for DeepSeek Harness — give text-only agents eyes and ears on video. A DeepSeek Harness (DSH) plugin that lets text-only LLM agents understand local video files. It provides two tools: Tool What it does video probe Cheap, instant metadata via ffprobe : container, duration, resolution, fps, codecs, audio tracks, subtitles. video analyze Content understanding: scene-change-aware frame sampling ( ffmpeg scdet ), optional ASR transcript (speech with timestamps), fused with any OpenAI-compatible vision model into structured evidence JSON. video ask Time-anchored Q&A : parses explicit time references ("at 3:20", "第2分钟") or locates relevant speech via transcript keyword matching, re-samples frames from the matched windows, and answers with grounded evidence (answer + confidence + supporting timestamps). v0.3.1. The plugin never locks you into a provider: vision and ASR are both OpenAI-compatible endpoints configured via baseUrl + model + key env var. How it works - Scene changes are detected with ffmpeg's scdet filter (ffmpeg ≥ 6.0). Videos without detectable cuts fall back to uniform midpoint sampling. - ASR is strictly additive : if asrApiKeyEnv is uns…

Category