HN user

akreal

40 karma
Posts0
Comments33
View on HN
No posts found.

WhisperX is not a model but a software package built around Whisper and some other models, including diarization and alignment ones. Something similar will be built around the Cohere Transcribe model, maybe even just an integration to WhisperX itself.

I also expected screenshots there, especially given the word "interface". Turns out, it's not about user interface (UI), it's about programming interface (kinda API). It allows calling window-related functions on Macintosh, X Window System, and Atari. So the resulting windows were looking like a native UI, I assume.

As far as I understand, the MMS TTS models are trained from scratch (section 7.1 of [1]), they do not employ any SSL models. So the OmniASR SSL models are not useful here.

What might be interesting is the newly released OmniASR data, because the MMS data, which was used for the MMS TTS, was never released.

Also, the OmniASR can be used to transcribe some untranscribed speech to train a TTS on it.

[1] MMS paper: https://arxiv.org/pdf/2305.13516

200ms Voice LLM 2 years ago

Cool!

HF Transformers is great for prototyping and research, but should not an interactive tool like this be based on something more speed-focused, like llama.cpp?

Any plans for languages beyond English?

I was curious about energy efficiency and took two samples from the linked MLPerf GPT3 results. H100 seems about three times more efficient than Gaudi2.

  256 Gaudi2 600W TDP: 256 * (442.578 / 60) * 0.6 = ~1133 kWh
  512 H100 700W TDP: 512 * (64.264 / 60) * 0.7 = ~384 kWh