Deep Dive into FFmpeg 8.0

https://www.rendi.dev/post/ffmpeg-8-0-part-1-using-whisper-for-native-video-transcription-in-ffmpeg

by dutzi • 8 months ago

40 11 8 months ago

rendi.dev

pinter695• 8 months ago

Author here, available for questions

cranberryturkey• 8 months ago

This is great I’ll have to give it a shot

lern_too_spel• 8 months ago

Whisper will hallucinate on audio segments that don't have any speech. VAD mitigates that. Expect worse results without it, especially on non-English audio.

trq01758• 8 months ago

"Lenovo laptop with Nvidia RTX 4040" 4060?

pinter695• 8 months ago

Correct. I fixed the typo

mikece2• 8 months ago

Is the point that you only need one tool -- ffmpeg -- to both generate transcripts as well as embed those into a video as opposed to having multiple tools?

pinter695• 8 months ago

This is a 3 part series, the first one discusses the new native whisper integration. And correct, for the first post - the point is to show that you can only use ffmpeg to transcribe and embed subtitles in a video

mikece2• 8 months ago

While there's appeal in having one tool do several things I'm more a fan of the traditional UNIX philosophy that a tool should do one thing, do it extremely well, and allow for chaining of several tools together to achieve a multi-step process.

radicality• 8 months ago

Do you know if it’s supported on Mac too, with whatever platform specific optimizations like running it on the gpu / with MPS ?

pinter695• 8 months ago

You mean Vulkan? In the blog post there is reference to all vulkan supported platforms

If you mean ffmpeg build with whisper - from memory I didn't see ffmpeg-builds for mac, so you will probably need to compile yourself

Deep Dive into FFmpeg 8.0

Related Stories