HN user

eginhard

100 karma

https://enno.xyz

Posts4
Comments34
View on HN

STT training data includes all kinds of "noisy" speech so that the model learns to recognise speech in any conditions. TTS training data needs to be as clean as possible so that you don't introduce artefacts in the output and this high-quality data is much harder to get. A simple inversion is not really feasible or at least requires filtering out much of the data.

I found it easy to get started with the very basics (e.g. recording of simple transactions) and I'm just reading up more over time on how to handle more complex things like splitting expenses with a partner or investments. Thanks to Python I was also able to customise my setup almost right away. There is also an Emacs mode for those who have already made that investment.

Recordings are force-aligned to the transcriptions anyway (using essentially a speech recognition system) to obtain phone-level alignments. You don't need explicit timing information beforehand.

Even if there is no background noise present, the quality is nowhere near that of professional studio recordings and would be very noticeable in the output.

Also, for traditional systems you need a lot of data from one speaker only, they can't take advantage of other speakers' recordings (although WaveNet does that now).

And "TV natural" might not be the style of natural you want from a TTS system.

From LSA/SVD you get a V x K matrix as well - that's exactly what the factorisation is doing.

The following two papers also go into detail about the mathematical similarities between LSA and neural embeddings and achieving similar performance with both:

Levy, O. and Goldberg, Y. (2014). Neural word embedding as implicit matrix factorization. https://www.cs.bgu.ac.il/~yoavg/publications/nips2014pmi.pdf

Levy, O., Goldberg, Y., and Dagan, I. (2015). Improving distributional similarity with lessons learned from word embeddings. http://www.anthology.aclweb.org/Q/Q15/Q15-1016.pdf