Meta says its new speech-generating AI tool is too dangerous to release

https://www.techradar.com/news/meta-says-its-new-speech-generating-ai-tool-is-too-dangerous-to-release

by homelesscodes • 3 years ago

29 51 3 years ago

whelp_24• 3 years ago

hmm, hasn't speech been spoofable before? Is there a major difference between this and video?

bdcravens• 3 years ago

Perhaps this defeats common security by voice systems, something that even good APIs like Eleven Labs cannot (I assume).

mistercow• 3 years ago

If so, they really should disclose that. At this point, voice recognition is just a security time bomb.

falcolas• 3 years ago

Nice marketing piece.

It’s too dangerous to release until enough buzz is built about how dangerous it is. Then it will be released (or “leaked”).

Tao3300• 3 years ago

Glad I'm not the only one who read it that way. It just smacks of "we are just way too good at what we do".

lvturner• 3 years ago

Not entirely sure why this was downvoted, it was my initial thought too (though I must concede I’m reading the comments before the article… which may give me my answer…)

camillomiller• 3 years ago

Well they’re good at copying, so they’re doing exactly that with the OpenAI marketing strategy

T3OU-736• 3 years ago

Next version of "You WON'T BELIEVE how DANGEROUS this Meta voice AI model is".

pier25• 3 years ago

I can't believe it's not butter! Sorry, ai.

tlogan2• 3 years ago

I 100% agree.

This article is a good promotional article to get people hyped about this new product. I must admit, it has sparked my interest. I'm eager to use it. At last, I won't have to deal with inconsistent voice actors for my wizard demos / support pages.

Surely, they may have to incorporate some safety measures, much like OpenAI has.

Avshalom• 3 years ago

So what, they set out to make a speech-generating AI tool and only after they finished it realized that they had made a speech-generating AI tool?

ulnarkressty• 3 years ago

The engineer/maker types sometimes fail to consider second order effects of whatever they are building if the coolness factor is high enough.

eloisius• 3 years ago

I would guess that “too dangerous” is both hype and also means it utters insensitive statements. Not that it’s primed to topple human society.

111111IIIIIII• 3 years ago

This is the only thing that makes sense.

ChildOfChaos2• 3 years ago

What do you mean it 'utters insensitive statements', this is not a chat bot, it's text to speech, it generates speech for exactly what you type in, the threat is that it can be trained to sound like exactly anyone and the potential for fake news etc is high.

The trouble is, this is already partly here and going to be increasingly available. For smaller startups that already do this like ElevenLabs, they have no reason to hold back, as it's there entire business, Meta can for now, but it's a fruitless exercise. These tools are coming.

Meta doesn't want the flack hurting or causing investigations into it's current business is more what it means, it will release the tool when others have released there's and it will no longer be the one to blame.

flangola7• 3 years ago

The FBI warning about malicious communication using loved ones' voices is very damning.

gonzo41• 3 years ago

Yeah they don't want to be blamed for more deep fakes and election interference. This sort of tech is a strategic risk to meta.

radq• 3 years ago

More like the engineers and scientists built it, but legal and PR think it's a bad idea to release. Which they're probably right about considering the reception Galactica got.

pharmakom• 3 years ago

What was the success criteria of the project? sounds like they did what they wanted and only after realised they didn't want to? why?

usrusr• 3 years ago

The goal of that kind of project in a huge tech org is to have capability in-house. So that they can play that game if they have to. Seems like they don't see themselves in a situation where their position without the tech would be worse than with the tech deployed. In some ways it's a bit like defense spending in a way, the most successful defensive army is the one strong enough to never get tested (in many other ways it's not)

toyg• 3 years ago

Their scientists were so preoccupied with whether they could, they didn't stop to think if they should. - Dr. Ian "Goldblum" Malcolm

ionwake• 3 years ago

Does anyone know who the leader is in affordable speech synthesis for hobby projects ?

Atm I am using Amazon Polly but am unsure if another offering is far superior. eleven labs is not affordable for my slightly serious level of job ting but aws and google seem well priced. Any other suggestions for a product I can try out?

mesmertech3• 3 years ago

https://github.com/neonbjb/tortoise-tts

Give this a try. You can run it locally if you have a good enough GPU. It is pretty slow at generation tho

tornato72• 3 years ago

I've tried this and it took ~5 minutes to generate 10 seconds of studio on my 3080. That just doesn't work if you're trying to generate an hour long podcast.

reckless• 3 years ago

Take a look at Piper. It's the tts solution used by the open source home automation project HomeAssistant. Produces decent quality speech in a couple seconds on raspberry pi class hardware.

candiodari• 3 years ago

I think this is the link. Not 100% sure though, is this correct?

https://github.com/rhasspy/piper

vgchh• 3 years ago

AI may be meta's only and real "grow out of newsfeed business" card. They should probably pivot, release their own version of ChatGPT/VoiceGPT, start charging for it and maybe rebrand (metai?). They are currently not being taken seriously, despite the chops they have, due to their stupid obsession with social and connecting people.

whimsicalism2• 3 years ago

Perhaps by consumers but among the AI industry/NLP I would view them as the top 3 most sophisticated companies. They have also gained a ton of respect from me for not joining in with the "closed" approach other companies have all now adopted.

daveguy• 3 years ago

They have, but when OpenAI started spouting "too dangerous to release" was about the same time they effectively dropped the "Open" part. I hope that is not the same for Meta. But wasn't Meta not releasing model weights except for a leak? In that respect are Meta and OpenAI already equivalent?

ajkjk• 3 years ago

Well they did that because they were in third, ie they had little to lose from 'commoditizing their competitors'

kidgorgeous• 3 years ago

Google also thought their AI was too dangerous to release....so chat-gpt released theres. See a pattern?

tornato72• 3 years ago

What would happen if only bad actors had access to LLMs, deepfakes, and stable diffusion models? They could convincingly fake image/video evidence and accompanying online support for disinformation, and most people would buy it.

But today, given the widespread availability of these tools, most people know what they're capable of, and when a meme photo of Donald Trump in handcuffs gets circulated nobody takes it seriously because they have seen dozens of fakes before.

So IMO if you really want to minimize the potential for your AI tool create chaos, release it to the public. Show people what it's capable of. Once people tire of memes of Obama spouting the navy seal copypasta, they will be prepared to call out disinformation generated by these tools as well.

tlogan2• 3 years ago

I believe everyone can understand this - and they will need to releases it to public. Surely, they may have to incorporate some safety measures, much like OpenAI has.

Anyway a fantastic promotional strategy to get people hyped about this new product. I must admit, it has sparked my interest. I'm eager to use it. At last, I won't have to deal with frustrating and inconsistent voice actors.

imagine99• 3 years ago

Adobe demoed a similar project as far back as 2016 called "Project Voco" [1] which was also called "too dangerous to release" at the time, even though it apparently still needed as much as 20 minutes of source material (vs. allegedly a mere 2 seconds here).

It was never heard from again afaik - even though Adobe is not known to shy away from an opportunity to increase revenue, so one cannot help but wonder...

[1] https://en.wikipedia.org/wiki/Adobe_Voco

jacknews• 3 years ago

lol, what BS.

We could release it, but then we'd have to kill you.

civilized• 3 years ago

They said the same thing about GPT-3. It's a marketing push.

Oh no, our incredible scientists made a thing that's simply too amazing and they're afraid it might take over the world! We just thought you might like to know how nice and ethical we're being with our immense power...

smusamashah• 3 years ago

It's demo wasn't very good compared with other state of the art tools available these days. You could hear so many glitches in audio.

Also OpenAI said the exact same thing about GPT but still released it. Meta's tool isn't worth hyping like that, not in its current state.

judge2020• 3 years ago

Its demo wasn't good even compared to the Adobe AI voice generation demo from 2016.

can16358p• 3 years ago

Whoever needs to use it at deep-state level for nefarious purposes will either access it or find something similar that does the same job.

I think it should be released so that the public can see what can be faked and they get used to the new paradigm of everything can be faked so they don't immediately believe in everything they see/hear.

TL;DR: bad guys will use similar tech anyway.

ez_mmk• 3 years ago

There was also a Google employee who said the same about Bard

taneq• 3 years ago

"Too dangerous to release" just means "we don't think anyone else has something better."

Just wait, if their next version of LLaMA beats GPT4 it'll be "too dangerous" otherwise it'll be released as soon as they think it's peaked.

theage• 3 years ago

Bell labs thought something similar about the first answering machine, fearing people would be held to their word resulting in fewer telephone calls.

Now big tech worries that "perfect" trust in their communication platform goes away, but the paradigm shift happens no matter what. Many governments already have the power to post as users on platforms and that should be cause to usher in this new trust calculation faster.

xnx• 3 years ago

Open AI used the same "too dangerous" marketing technique with GPT2 in 2019. https://www.theguardian.com/technology/2019/feb/14/elon-musk...

bradgranath• 3 years ago

Can't fix facebook. Can't find a reason to get the whole world to go to work in VR. Can't release their apocolypse Robot.

What else can't they do?

Meta says its new speech-generating AI tool is too dangerous to release

Related Stories