I've been playing around with local LLMs to see how well they work as part of a video RAG platform.
The platform detects shots, transcribes speech, describes what is happening on screen, identifies events and groups everything into scenes.
Those scenes are then indexed in Qdrant, allowing you to search for a specific moment, ask a question about what was said or shown, and view the answer alongside the relevant timestamp.
The pipeline uses TransNetV2, faster-whisper, SigLIP 2, Qwen2.5-VL, Marlin-2B, Qwen3 and BGE-M3, with a simple Vue interface.
Thirty public-domain movies, cut down to 15 minutes each, were processed locally on an 8 GB laptop GPU. Average processing time per movie was 13.5 minutes, or 0.9× real time.
For a local system, the results are pretty good. There are a few hallucinations, but these may be due to the quality and limited quantity of the source material.