Mozilla Common Voice Adds 16 New Languages and 4,600 New Hours of Speech

https://foundation.mozilla.org/en/blog/mozilla-common-voice-adds-16-new-languages-and-4600-new-hours-of-speech/

by heyhillary • 5 years ago

728 170 5 years ago

foundation.mozilla.org

satya71• 5 years ago

The top five languages by total hours are English (2,630 hours), Kinyarwanda (2,260), German (1,040), Catalan (920), and Esperanto (840)

Some unusual suspects among the top languages, there!

ftyers5• 5 years ago

That's what happens when people have the opportunity and tools to support their own languages and not just rely on hand outs from big tech :)

Anon1096• 5 years ago

Esperanto is a hobby language for upper-middle class people in developed countries. It isn't anyone's "own language".

1-6• 5 years ago

You have a point there. I've been disappointed that Korean has been stuck in the 'In Progress' state. The Korean tech giants already have APIs to do common speech recognition. I hope more Korean grassroots efforts focus on tools that are open and accessible so it can be built scalable and better.

umeshunni2• 5 years ago

Ah yes, major world languages with 10s or 100s of millions of speakers (Bengali, Korean, Malayalam) are ignored or are perpetually stuck "in progress" while hobby languages like Esperanto are supported.

yorwba4• 5 years ago

The project seems to have some serious government backing in Rwanda: https://digitalumuganda.com/

junon• 5 years ago

Historically not been the biggest fan of Mozilla but I really, really love this project. I'm glad they're keeping it alive.

ftyers5• 5 years ago

One of the most noticeable additions in my opinion is Guarani, the first Indigenous language of the Americas to be added. Indigenous languages are extremely poorly supported and forgotten by all of the major platforms and companies, and it's great to see one getting the attention they deserve. (Disclaimer: I was involved)

neartheplain• 5 years ago

Whoah, 6.5 million native speakers! That's several orders of magnitude more than I was expecting. It's also significantly larger than the native-speaking populations of languages like Catalan, Basque, or Romansh, which might be more familiar to North Americans or Europeans.

djoldman• 5 years ago

Or, 20x more than Icelandic:

https://en.wikipedia.org/wiki/Icelandic_language

andrepd• 5 years ago

It is one of the official languages of Paraguay (along with Spanish), where it is spoken by the majority of the population, and where half of the rural population is monolingual.

Wow, I had no idea

kspacewalk2• 5 years ago

There are a number of Native American languages that have numerous speakers, but until recently have been marginalized, repressed and ignored (and some to this day). Guarani is the most numerous, but also Quechua, Nahuatl, and the various Mayan languages (spoken by around half of Guatemalans, and another 2.5 million Mexicans).

victorlf• 5 years ago

Catalan has about 10 million speakers.

runarberg6• 5 years ago

As an Icelander I am always really impressed with how well my language—a language spoken by a few hundred thousand people worldwide—is supported on various platforms and technologies. This is probably in no small part thanks to active participation by native speakers and even some government funding.

However I at the same time I’m also deeply disappointed by the lack of support for Iceland’s closest neighbour’s language—Greenlandic—which is an indigenous language, the sole official language of an autonomous country.

matsemann• 5 years ago

I saw the same when I was younger for Norwegian. Bokmål is the most commonly written form of Norwegian, but New Norwegian is used by about ~15%. Most software included Bokmål support, but you could bet some hardcore user of New Norwegian had made a language pack available as well.

simongray• 5 years ago

However I at the same time I’m also deeply disappointed by the lack of support for Iceland’s closest neighbour’s language—Greenlandic—which is an indigenous language, the sole official language of an autonomous country.

I'm not sure "autonomous country" is an accurate description of what Greenland is. It is - for all intents and purposes - a devolved region of Denmark. It is still way too reliant on economic aid to be able to be independent and, honestly, probably couldn't exist as a developed nation without a patron (Denmark) or without selling its land/resources to a great power (USA, China). And the population is only 1/6 the size of Iceland's and is very dispersed on a massive arctic island, with most people living in tiny isolated villages by the coast.

With that in mind, you wouldn't expect great language support unless the Danish state steps in and spends some serious dough on it. I actually work on Danish language technology at the University of Copenhagen and let me tell you something... the Danish state hardly spends any money on Danish language resources either. We envy the kind of funding that researchers in countries like Iceland and Norway have access too.

danShumway3• 5 years ago

I don't really have anything of substance to add here, but I'm very happy to see Mozilla continuing to put effort into this, happy to see effort being put into broadening the support beyond just English and major languages, and I'm grateful for the work that people (inside and outside of Mozilla) have already put into getting the project this far.

orra3• 5 years ago

Indeed, it's great to see open data corpuses expand.

mgarciaisaia• 5 years ago

You arguably have something of substance to add - you can help improve the datasets by speaking or validating phrases in the project's website

https://commonvoice.mozilla.org/

There are many languages available to pick from.

S5yDyAk3XoQH5• 5 years ago

Everyone has something to add. Go read stuff on their site in the languages you know. If people would actually do it for a few months all languages would have been done years ago.

_gtly• 5 years ago

A direct link to where you can donate your voice here: https://commonvoice.mozilla.org/en

jalopy• 5 years ago

Going along with this: What are the latest and greatest open source speech-to-text models and/or tools out there?

Would love to hear from experienced practitioners and a bit of detail on the experience.

Thanks HN community!

orra3• 5 years ago

Mozilla announced Deep Speech[1] around the same time as Common Voice.

Mozilla Deep Speech is an open source speech recognition engine, based upon Baidu's Deep Speech research paper[2].

Unsurprisingly, Deep Speech requires a corpus such as... Common Voice.

[1] https://github.com/mozilla/DeepSpeech

[2] https://arxiv.org/abs/1412.5567

rasz5• 5 years ago

They killed this after Nvidia grant.

zerop2• 5 years ago

Vosk is my favourite. I have used deep speech too. Vosk works better.

nshm3• 5 years ago

Thank you. I deeply appreciate you mention our efforts. We spend quite some time and knowledge to build accurate speech recognition. Not that easy to get as much mentions as Mozilla, so we are thankful for every single one!

thom• 5 years ago

Same question for text-to-speech!

kcorbitt• 5 years ago

I've had good results with https://github.com/flashlight/flashlight/blob/master/flashli.... Seems to work well with spoken english in a variety of accents. Biggest limitation is that the architecture they have pretrained models for doesn't really work well with clips longer than ~15 seconds, so you have to segment your input files.

mazoza3• 5 years ago

https://github.com/coqui-ai/STT

jononor3• 5 years ago

Have used VOSK a bit recently. The out-of-the-box experience was great compared to earlier projects (looking at you Kaldi and Sphinx...). Word-level audio segmentation was one usecase, https://stackoverflow.com/a/65370463/1967571

woodson2• 5 years ago

Vosk is built on Kaldi.

stegrot8• 5 years ago

Kdenlive supports automatic subtitles created with VOSK now btw. This makes it a lot more accessible for non-tech folks.

woodson2• 5 years ago

NVidia NeMo: https://github.com/NVIDIA/NeMo

blackcat201• 5 years ago

I created edgedict [0] a year ago part of my side projects. At that time this is the only open source STT with streaming capabilities. If anyone is interested the pretrained weights for english and chinese is available.

[0] https://github.com/theblackcat102/edgedict

lazyresearcher• 5 years ago

Kaldi and DeepSpeech both support streaming, right?

donhaker• 5 years ago

Let's take the time to appreciate the effort of Mozilla. To add new languages with others came from the minorities, we can't deny that they are continuously putting effort into the community.

Jnr• 5 years ago

The great open source community around Mozilla helps a lot.

When I did not see my own language in the list a year ago, and I had no clue how to get it there, I reached out to my university contacts that I know used to translate Firefox years ago.

With their help we quickly translated the whole common voice site (it was a prerequisite to start contributing a language) and provided first sets of text to start contributing.

In about a week we started contributing voice for a new language. The Common Voice project is awesome and very well made.

fisxoj• 5 years ago

If anyone has interest contributing, I've found this app for Android makes it very easy! https://www.saveriomorelli.com/commonvoice/

nmstoker4• 5 years ago

Why on Earth would anyone use an app for this when mobile browsers work perfectly well for adding audio to Common Voice?

We could possibly give the developer the benefit of the doubt that they're not doing anything inappropriate with the data but frankly why pass your data through a third party that's not part of the project.

And why install an app requiring access to your shared local storage? The GitHub repo claims the website an animations are slow which sounds like BS to me. It works fine on a five year old phone I use for submitting.

Just contribute here if you're so inclined, much more sensible:

https://commonvoice.mozilla.org/en

totetsu2• 5 years ago

because mozilla fired all the cv team, and the app is under active development?

commoner• 5 years ago

The unofficial CV Project Android app is entirely open source and available on F-Droid:

https://github.com/Sav22999/common-voice-android

https://f-droid.org/packages/org.commonvoice.saverio/

stegrot8• 5 years ago

The app has a few nice features the website doesn't have, such as changing the speed during validation. It always surprises me as well, but many people hate to use web apps on mobile. I don't really know why, they simply ask for an app and refuse to use a browser.

rasz5• 5 years ago

Whats the point when they killed DeepSpeech in exchange for adapting closed Nvidia thing?

https://venturebeat.com/2021/04/12/mozilla-winds-down-deepsp...

https://blog.mozilla.org/en/mozilla/mozilla-partners-with-nv...

$1.5mil for shutting down open source initiative, almost half of CEO salary right there.

moralestapia• 5 years ago

Lol, these guys sell themselves for peanuts.

jononor3• 5 years ago

What closed NVidia thing did they adopt? I don't see any evidence of that here.

rasz5• 5 years ago

Im sure they signed on adopting "something", otherwise it would be receiving $1.5 million grant for closing open source initiative. $3 million a year lawyer wouldn never be this blatant.

option• 5 years ago

https://github.com/NVIDIA/NeMo which is open source, Pytorch based and regularly publishes new models and checkpoints.

stegrot8• 5 years ago

Deepspeech is still alive in a way, the team founded the company coqui.ai after the Mozilla layoffs and they keep everything open source.

mazoza3• 5 years ago

I know the old speech team continues as Coqui https://github.com/coqui-ai/

tmalsburg23• 5 years ago

About their TTS system: "These models provide speech synthesis with ~0.12 real-time factor on a GPU and ~1.02 on a CPU." The quality of the samples is really impressive but, wow, but isn't this computationally too expensive for many applications?

jononor3• 5 years ago

Open-source speech recognition is doing pretty good with projects such as VOSK, Athena, ESPNet and SpeechBrain. These days models are the easy part of ML, and data is the hard one. So for Mozilla to focus on Common Voice over DeepSpeech seems reasonable.

tkinom• 5 years ago

Would one use the youtube as training date?

Especially for the videos with Close Caption....

As simple as extracting the Audio and CC text?

hkt4• 5 years ago

Having an open corpus means that researchers building the next thing in voice research - which may or may not follow DeepSpeech - have something to work with. This is enormously important and their change of direction lets a thousand flowers bloom. Meanwhile, their partnership with Nvidia provides a fertile ground to prove the value of the open corpus in action. Nvidia get access to Mozilla's (presumably superior) ability to build said corpus, while Mozilla lay the foundations for others to contribute work in the open. It is a great example of comparative advantage, and a win win choice, IMO.

rasz5• 5 years ago

So in other words we provide data for free to Mozilla, and Mozilla turns around and sells it for millions to Nvidia to fund ... not open source, they killed that so umm ee, to fund ceo salary?

pkz• 5 years ago

Openly licensed speech data for smaller languages is great! I hope as many as possible contribute in order to get better representation across ages and pronunciation. In the end, this may be what is needed for the hyperscale companies to support speech assistants in more languages?

tsjq• 5 years ago

nice. !

news from the past about this :

Initial Release of Mozilla’s Open Source Speech Recognition Model and Voice Data : https://news.ycombinator.com/item?id=15808124

Mozilla releases the largest to-date public domain transcribed voice dataset https://news.ycombinator.com/item?id=19270646

bravura• 5 years ago

Is anyone aware of classification (e.g. word prediction) datasets for low-resource and endangered languages?

If so, we would like to use it for the HEAR NeurIPS competition: https://github.com/microsoft/DNS-Challenge/tree/master/datas...

The challenge is restricted only to classification tasks, and sequence modeling like full ASR is unfortunately beyond the scope of the competition.

LoriP2• 5 years ago

Tips & Tricks incoming... I find that if I can't sleep and want something that's kind of useful to do without getting too involved, contributing to common voice is a great way to spend half an hour and relax/forget whatever it is I was churning about. I would recommend it for that, plus it's a great project. Both listening and voicing...

dabinat3• 5 years ago

Common Voice is a great project that I’m glad Mozilla kept alive.

One problem is that data for speech recognition needs to be extremely accurate (i.e. the speech matches the transcript perfectly) and the human review process is infallible and there are quite a number of bad clips that made it past the review process (to be fair, Mozilla provides no official guidance to reviewers or recorders).

Plus in the early days, they were recording the same small sentence pool over and over again, so the first 700 hours or so are duplicates.

I hope there will be efforts in the future to clean up the existing dataset to improve its quality.

ma2rten4• 5 years ago

Why does data for speech recognition need to be prefect. That's certainly not the case for other machine learning applications. Can you train the less clean data and fine-tune on a clean subset?

dabinat3• 5 years ago

Well that was kind of my point: you need to manually figure out what’s clean and what isn’t.

stegrot8• 5 years ago

Here are some draft guidelines for validation that have been translated a lot: https://discourse.mozilla.org/t/discussion-of-new-guidelines...

But you are right, the process has some flaws. Maybe we can review the dataset automatically on some common errors, once an STT system is ready for a language?

The only other option I can think about is a validation process that includes more people per sentence. Right now, only two people validate a sentence, and if they disagree a third person decides. We could at least double check sentences with one "no" vote one more time.

dabinat3• 5 years ago

The community guidelines are good but they’re hidden away on the forum. I was asking them for years to just make those the official guidelines and link them prominently on the CV site but they never did.

However, Hillary, the new community manager, seems good and she’s making a lot of positive changes so hopefully this will be addressed soon.

Long-term the best approach may be some kind of user onboarding before they can record / validate.

lunixbochs• 5 years ago

I'm an ASR researcher shipping high quality English models trained on limited resources, and while I've needed to include other datasets to make the model more robust to different kinds of text, Common Voice is a substantial part of my training process. I did not do any manual transcript accuracy cleanup. Most of my automated cleanup was done with very basic (low quality) models. My latest models trained this way are competitive with e.g. Google or Apple English speech recognition accuracy.

I'm going to disagree that there's a universal need for perfect training data in ASR. I'm sure it helps with some model types and training processes, but it simply hasn't been a factor in my use of Common Voice (English). I'll also note my best model can hit around 10% WER on Common Voice Test without any language model, which is better than any public numbers I've seen posted for it so far (I'm not even using a separate transformer decoder or RNN decoder layers for this number, just the raw output of CTC greedy decode).

None of the above even factors in techniques like wav2vec and IPL (iterative pseudo labeling) with noisy student, which suggest you can hit extremely competitive accuracy with very little correctly labeled data. These techniques are the underpinnings of the current state of the art models.

johnnyApplePRNG• 5 years ago

Just tried rating some of the English voices and I am conflicted.

Most of them were definitely speaking English, but in an Indian intonation that I was barely able to understand coming from an English as a First Language country.

Some of them were reading words syllable by syllable, which is definitely English, but I would hate to have to listen to an ebook or webpage read aloud to me in that manner.

By clicking yes am I training the system to speak English with an Indian intonation?

Should I click no, not English?

Should/does english even have a "proper" intonation?

ma2rten4• 5 years ago

I think this dataset is mainly for speech recognition and not text to speech. Speech recognition should be able to recognize as many different accents as possible.

jturpin3• 5 years ago

Wow you're right. This is conflicting as many of the words are not pronounced properly at all. Maybe it doesn't matter to the accuracy of the speech-to-text system, but it feels like training it with bad data.

ohgodplsno• 5 years ago

Different accents isn't bad data. Your vision of the world of "english is only spoken with an american accent" is what leads to horrendous speech recognition APIs, like Google's.

If your ML model can't handle multiple accents, it is worthless.

humanistbot2• 5 years ago

That's the point! When the postal service has to OCR mailing addresses, they need to do the messy scribbles more than the professionally printed labels.

marc_abonce• 5 years ago

From https://commonvoice.mozilla.org/en/criteria

Varying Pronunciations
Be cautious before rejecting a clip on the ground that the reader has mispronounced a word, has put the stress in the wrong place, or has apparently ignored a question mark. There are a wide variety of pronunciations in use around the world, some of which you may not have heard in your local community. Please provide a margin of appreciation for those who may speak differently from you.
On the other hand, if you think that the reader has probably never come across the word before, and is simply making an incorrect guess at the pronunciation, please reject. If you are unsure, use the skip button.

magicalhippo• 5 years ago

Common Voice is not for generating speech, it's for detecting speech.

So don't worry about weird intonation as long as they correctly pronounce the sentences, that way even more people can enjoy the fruit of this labor.

usr1106• 5 years ago

I think the reality is that there are more speakers of bad English than native speakers. I speak 2 foreign languages (including English) daily and 2 others occasionally. I know I make mistakes in all of them. In English I don't think I make a lot of pronunciation mistakes (there are some mistakes in grammar for sure). In Finnish I make a lot of pronunciation mistakes, although I speak better than many other non-native speakers. How much that really hurts understanding I have no idea. The amount of misunderstandings between humans does not seem to vary greatly between those languages or even my mother tongue.

Text to speech should work correctly. But speech recognition should tolerate even clear mistakes. Of course not for the price of misunderstanding correct pronunciation.

fareesh• 5 years ago

Is voice transcription accessible to mere mortals yet?

I have tried pretty much every API offered by big tech, and also various open source models. All of them seem to have incredibly high word error rates. This is mostly for conversations with various Indian accents.

nshm3• 5 years ago

Did you try Vosk Indian English model? It is specifically built for Indian accent English

https://alphacephei.com/vosk/models/vosk-model-en-in-0.4.zip

In case you want more accuracy you can share a file with an example, we can take a look on how to make the best accuracy.

For Indian ASR it is also worth to mention recently introduced Vakyansh project which builds model for major Indian languages:

https://github.com/Open-Speech-EkStep/vakyansh-models

alpb• 5 years ago

This may be off-topic but: What's the relationship between Coqui (an OSS TTS startup) https://coqui.ai/about and Mozilla? I recall that the project at one point was called mozilla/TTS (https://github.com/mozilla/TTS/) and now I see that has a fork in the startup's own repo (https://github.com/coqui-ai/TTS). Presumably Common Voice is used to train mozilla/TTS and other OSS TTS solutions?

ftyers5• 5 years ago

Common Voice is mostly used for STT not TTS. TTS requires single speaker, clean audio. STT requires multi speaker, noisy audio.

say_it_as_it_is2• 5 years ago

"The top five languages by total hours are English (2,630 hours), Kinyarwanda (2,260) , German (1,040), Catalan (920), and Esperanto (840)."

How did they get almost as much training for Kinyarwanda as they have English?

stegrot8• 5 years ago

The German Federal Ministry for Economic Cooperation and Development supported this language: https://www.bmz.de/de/aktuelles/intelligente-sprachtechnolog...

say_it_as_it_is2• 5 years ago

Interesting! There's a market for this kind of audio data entry? What was the total cost for that many hours? The English data was entirely volunteer driven, correct? Maybe it's worth funding the English corpus for the additional hours needed to reach the sweet spot?

nyx-aiur• 5 years ago

I love the datasets but they are still way to small especially for exotic languages.

Edman2742• 5 years ago

I'm guessing that of the 4,600 new hours of speech, maybe 4,100 of those hours are of men's voices and 500 hours are of women's voices, yeah?

LoriP2• 5 years ago

To be fair not sure that's the best guess :) there seem to be more female voices than men to me. Anyhow, I'd wager there's at least a 50:50 mix.

Edman2742• 5 years ago

I probably should've looked this up before I decided to comment but at least according to this:

https://commonvoice.mozilla.org/en/datasets

The ratio of male to female tagged voices in the English dataset is 45 percent male to 15 percent female. (The remaining 40 percent is untagged.) Odds are good that the ratio is closer to 75 25 than 50 50, at least by hours of recorded audio.

heyhillary2• 5 years ago

Thanks so much for sharing your comment. Gender equality in participation in Common Voice, is something we really want to improve and champion. As part of the Kiswahili Language community engagement, our team are implementing a gender action plan that includes both participation and use cases for the dataset. We hope to consult, adapt and replicate gender inclusion that has been done by community members and gender action plan to improve representation and involvement of all genders in open source projects such as Common Voice.

olejorgenb• 5 years ago

I find the recording UI a bit annoying. They make it unnecessary hard to re-record a clip. Re-recording the previous clip is likely to be a common thing to do. Instead of providing a shortcut for this, they have shortcuts for re-recording each of the individual 5 clips..

It's also impossible (?) to undo a clip. Eg.: If I've already recorded 3 clips and mistakenly begin a clip I simply can't pronounce correctly, there's no way of removing that clip without discarding the whole set. (EDIT: it is possible by re-recording that clip and pressing skip)

Vinnl• 5 years ago

Re-recording a clip is very rare for me. Keep in mind that it's supposed to emulate real-world conditions, with all its messiness.

SilverRed• 5 years ago

Yeah I think minor mess ups as long as the words are correct is actually good. As well as a bit of background noise. Problem is if Moz builds up a dataset of pure recordings and someone tries to use it but they are in a noisy room and the ML was never prepared for this.

arghwhat• 5 years ago

People seem to speak extremely mechanically in these samples, which I suspect may lead to training bias against native as speech if used.

I think it should be explained that one should speak naturally when reading the lines.

stegrot8• 5 years ago

Many people are also speaking very mechanically when they use a voice assistant, though ;) I believe we need a good mix, but telling people to speak a little more naturally certainly would help.

Mozilla Common Voice Adds 16 New Languages and 4,600 New Hours of Speech

Related Stories