OpenAI says it can clone a voice from just 15 seconds of audio

https://www.engadget.com/openai-says-it-can-clone-a-voice-from-just-15-seconds-of-audio-190356431.html

by notRobot • 2 years ago

47 70 2 years ago

ShamelessC2• 2 years ago

Isn't that true of approximately every voice cloning model released lately? I've seen as few as 3-5 seconds as a matter of fact...

rany_• 2 years ago

They're just saying anything to bump up their value.

ShamelessC2• 2 years ago

Certainly what it seems like. They have a model they can't release without backlash, but still want the credit and don't want to wait to receive accolades until they can figure out how to prevent abuse (which I assume is just impossible).

vouaobrasil8• 2 years ago

FTA:

OpenAI says they see this technology being useful for reading assistance, language translation and helping those who suffer from sudden or degenerative speech conditions.

Of course they would. They are manipulating others by appealing to accessibility. Who can argue with that? Accessibility has become the "but think of the children" appeal of the AI world. Yes, it can help those who suffer from degenetative speech conditions, but it can also deceive and push propaganda to whole new heights.

This sort of manipulation is characteristic of antisocial personality disorder. In fact, if you look up the symptoms of that disorder, technology companies like OpenAI exhibit quite a lot of them...

e40• 2 years ago

Don’t a lot of companies exhibit symptoms of ASPD?

vouaobrasil8• 2 years ago

That is true, but there are two replies:

(1) First, most large companies are essentially tech companies in some way, at least in that they exist to advance technology. Therefore, your observation is good reason to abolish large companies.

(2) (Direct) tech companies often exhibit one symptom that other companies don't, at least in not such a strong way. Tech companies lie more often directly to the public: car companies, drink companies, etc. don't lie about making the world a better place, they just try to appeal to basal desires. Tech companies actually are deluded enough to think they are making the world a better place, and use manipulation to convince us that this is true.

mysterydip• 2 years ago

"We have had the approval from the BBC to use the AI generated voice so we won’t need Sara anymore.”

https://www.hollywoodreporter.com/business/business-news/ai-...!

jsheard• 2 years ago

As a wise man once said: "drunk driving may kill a lot of people, but it also helps a lot of people get to work on time, so, it;s impossible to say if its bad or not"

vouaobrasil8• 2 years ago

Sounds like a fool. We have the ability to weight costs and benefits. We have advanced linear models and computers. We have the math to split the fucking atom, but when it comes to even thinking about the social costs of technology, suddenly everyone is impotent. What a joke.

lewhoo• 2 years ago

They are manipulating others by appealing to accessibility.

I'd argue voice cloning isn't really crucial to accessibility. I mean, this is just cosmetics. I'd like to hear opinions of people who actually can benefit from this and if it is really such a "game changer" for them.

vouaobrasil8• 2 years ago

Indeed, I agree. And before I hear from them, I would like to make sure that it wasn't some tech company that provided all the funding and technology in return for a PR piece as is quite common in the tech world.

fragmede3• 2 years ago

So Apple already did this, for that reason though.

https://www.aarp.org/home-family/personal-technology/info-20...

vouaobrasil8• 2 years ago

Yes they did. That is typical of tech companies: it's amazing that people actually believe it. But that's our societies attitude: new tech is an improvement. It's a faith stronger than most people have in their god(s).

bruce343434• 2 years ago

I think it's plausible that it can roughly clone the timbre that my exact vocal and sinus geometry produces, then apply that to its internal representation of a "baseline human voice". However, I don't believe that it can capture exactly every nuance in articulation, emotional expression, intonation. I simply can't imagine even a voice actor iterating through "the full range" in merely 15 seconds. And what about mannerisms, like pronouncing "you all" as "y'all"? Ending every sentence by affixing the "uh" sound to the last word-uh? Or how they sound a sigh or laugh?

In Dutch for instance, there's 3 ways to pronounce the R, and everyone has a certain combination of when they use which kind.

jstummbillig• 2 years ago

Sure. Edge cases will be infinite.

xbmcuser• 2 years ago

A lot of people miss the forest for the trees. They can clone a voice using 15 seconds of audio. So how good the clone will be when they have 10 min, 30 min or larger sample size. And it will probably keep getting better

marcusverus• 2 years ago

And what about mannerisms, like pronouncing "you all" as "y'all"?

In the US this is highly determined by location, to the point where the New York Times even built a cool quiz [0] that can guess where you're from pretty accurately based on a few dozen vocab questions. So there's a dataset out there that would allow you to handle most of this variance by simply plugging in a ZIP code for the person you're spoofing!

[0]https://archive.is/yodpA

criddell• 2 years ago

Does it really matter if they have not produced a perfect clone but merely a convincing clone?

numpad0• 2 years ago

It's often not necessary to sample all keys on a piano to make a passable piano sound generator, though that will have its own limitations.

A lot of these generators rely heavily on the base model and are worse at generating e.g. Dutch accented English and/or Dutch accented Dutch. But it works great for e.g. Californian accented English with unique voice.

verticalscaler• 2 years ago

I'll speculate "what is really happening" - Think of a voice like a musical instrument.

This learned to be a Shazam for human voices instead of songs. And just like you can figure out by ear a snippet of music was made on a Yamaha DX7, seems 15 seconds is enough to narrow it down to a reasonably small set of vocoders that can recreate the sample given.

23B1• 2 years ago

Altman and OAI – and their investors, more importantly – have already proven over decades and decades that they're in it for the money and at any cost to people including their own paying customers.

This generation is making the same can-kicking mistakes as their parents and nobody wants to admit it.

nonoesp2• 2 years ago

When the product officially rolls out there will be a “no-go voice list” that detects and prevents AI-generated speakers that are too similar to prominent figures.

Should they then let anyone register their "voice id" to prevent others frok generating similar voices?

What if your voice happens to resemble the voice of a "prominent figure?"

testacpwoek6• 2 years ago

It's insane that this protection exists for "prominent figures" but nobody else. The same damage that can be done to prominent figures can be done to regular people, they acknowledge the damage can be done, and yet they're still rolling this out? The more OpenAI does the more it's clear they don't give a single damn about the consequences of their technology.

xandrius• 2 years ago

That's what for-profit and investors do to morality.

zoky• 2 years ago

The “Open” in OpenAI refers to, uh, a certain part of your anatomy…

MrNeon• 2 years ago

It is very obvious that the amount and type of damage is not the same if you clone any random person's voice or clone a political leader's voice.

It is not the same damage.

mtillman• 2 years ago

It reminds me of the new EU law being implemented in that it eliminates the use of ai for private companies but explicitly allows it for law enforcement. Both are methods of consolidating power. Safer to be open for everyone and unrestricted than explicitly protecting the few.

Shawnj22• 2 years ago

It’s a little tricky because if someone cloned your voice first they could pretend they were you and stop you from using “their” voice, then do nasty stuff with it. This way they can still do that, but they can’t prevent you from using your own voice unless you have to upload an ID with your voice or something. It’s much easier to enforce this on known celebrities or otherwise well known people than the general public.

nonoesp2• 2 years ago

Descript acquired Lyrebird years ago and already have a product that can synthesize your voice.

They require training data longer than 15 seconds, which could lead the out out to resemble more the actual voice.

I've seen weird behaviors where the AI voice forces a British accent to pronounce certain words which I don't have.

Descript also uses voice synthesis to regenerate edited portions of conversations with a noticeable cut to smoothen the transition, which is pretty useful.

sherburt3• 2 years ago

I would imagine if I lost my voice that I would prefer a standard TTS voice as opposed to paying a subscription to have an unholy digital facsimile voice that OpenAI owned.

lr4444lr• 2 years ago

First text, then images, soon video and speech.

It feels like OpenAI is mostly concerned with developing proofs of the untrustability of every digital medium, creating a convincing case either for doing everything in person again, or necessitating cryptographic signatures on absolutely everything.

reaperducer2• 2 years ago

It feels like OpenAI is mostly concerned with developing proofs of the untrustability of every digital medium

Agreed. I'm still waiting for the part of AI where it's supposed to benefit mankind.

So far, it's 10% entrainment, 20% employment elimination, and 70% crypto-grade hype.

The next AI winter can't come soon enough so the tech industry can get back to doing useful things.

Asraelite• 2 years ago

20% employment elimination

How is this not benefiting mankind?

If you're concerned with unequal distribution of the generated wealth then that's a political problem, nothing to do with AI.

leovailati• 2 years ago

It feels like OpenAI is mostly concerned with developing proofs of the untrustability of every digital medium

Which, to me, makes sense. Once the underlying technology exists, a malicious actor would not think twice before developing tools of deceptionlike those. It makes sense that OpenAI would work on that "in the open" to demonstrate that we now need to be skeptical of audios.

anon3738393• 2 years ago

In some ways, this just feels like a reversion to the mean for humanity. “Don’t believe everything you read” is a maxim as old as print, and it’s good advice. If people ultimately learn that videos and recordings can’t be trusted, is that a huge loss? I’m not sure.

Of course, the road from here to there may be bumpy...

fragmede3• 2 years ago

scammers have already used this technology to convince people that someone they know needs money for some reason

mewpmewp2• 2 years ago

In which cases anyhow would they have been a trustworthy content. You could easily take real videos out of context which people were already doing to mislead.

Ultimately you should reason and investigate deeper to hold somewhat accurate opinion on anything.

Most media and news out there already is where the agenda comes first and then it is about compiling cherry picked content from wide variety of content as evidence for that agenda. And you can find content to support any agenda already.

It is a simple algorithm of:

1. We want to prove X.

2. From millions of datapoints we pick the 100 that support X the most.

3. We write an article on it.

4. Most of our readers already agree with X so they will be happy about it. No need to go deeper.

5. Anyone who doesn't agree would probably not read our platform anyway.

Shawnj22• 2 years ago

The answer we see is that if you put AI generated stuff in places people don’t look at too carefully they won’t care because they won’t notice. I’m not sure that’s great precedent

testacpwoek6• 2 years ago

In some ways, this just feels like a reversion to the mean for humanity.

No it doesn't, this is ridiculous. All of you guys with your "this is just like the printing press" arguments are so intellectually lazy, think a little more about the problems of scale and speed and the way information spreads in the modern world.

tomjen3• 2 years ago

Honestly cryptographic signatures being more widespread does not seem a bad thing to me.

Obviously not everywhere: I don't need a reddit worldnews comment or a recipe for blueberry muffins signed.

Turing_Machine• 2 years ago

OpenAI is mostly concerned with developing proofs of the untrustability of every digital medium, creating a convincing case either for doing everything in person again, or necessitating cryptographic signatures on absolutely everything.

Am I alone in thinking that this would not be a bad thing?

notRobot2• 2 years ago

I am not sure why this is being developed. The potential for misuse is so much greater than any possible beneficial use cases I can imagine. Text to speech is cool, why do you have to introduce technologies that allow for voice cloning, which will inevitably be used (more than it already is) to commit fraud and hurt people?

Society is not ready for all of these "intelligences", heck, we can't even figure out what to do about all the drawbacks of social media and we've had it for decades.

vouaobrasil8• 2 years ago

I am not sure why this is being developed. The potential for misuse is so much greater than any possible beneficial use cases I can imagine.

It's being developed because for decades, tech companies have touted the supposed benefits of new technology and hidden the costs. It's been an avalanche effect of greater and greater technology for greater and greater costs. Only now, the costs are so great that people are starting to wake up.

When we look back in time to see the devastation wrough by technology, we won't look at AI as the starting point. It will be the smartphones, the 4G internet, and the 8K video that we so blindly accepted without ever considering the immense changes to society that they implied. AI may be the start of the end of reasonable society, but it is only the APEX of a phenomenon that has been happening ever since we accepted fossil fuel use without considering the implicates of climate change. It's an entire societal attitude.

MyFedora• 2 years ago

I feel like I'm missing smth here?

Voice cloning has a good benefit to drawback ratio compared to many technologies. Threat actors with enough resources could always clone your voice. Everyone can hire voice actors. Costs more time and money sure, but it was always possible.

Beneficial use cases:

* Accessibility: Give people that lost their voice their voice back.

* Personalized Digital Assistants: More humane voice.

* Voiceovers: Sick content creator can let the AI do the voice, filmmakers can fix mistakes in post and so on and so forth.

* Language Learning: Learn an accent by mimicking how the AI voices yourself in the target accent.

* Audiobook Narration: Make a good audiobook out of a textbook.

* Preservation of Cultural Heritage: Preserve an accent that's about to die out.

* Corporate Training: Believe it or not, a robotic voice sucks to listen to after a while.

* Interactive Gaming: Entire virtual DND campaign based on a prompt with AI generated world, story, and characters drawn and voiced by AI with quality control by AI.

Now compare that to guns, tanks, flamethrowers, chemical weapons, atomic bombs, hydrogen bombs, killer drones and bombers. Yeah uh... death. Made to kill people.

notRobot2• 2 years ago

You are indeed missing something: you don't need machine powered instant voice "cloning" tech to have good artificial voices, which would do basically everything you're talking about.

Voice cloning has always been technically possible and so has fake videos but lowering the barrier of entry to voice cloning and deepfakes has very real implications for society that we're not remotely equipped to deal with.

Nothing is believable anymore. Stuff like this will only become more commonplace:

https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-ho...

https://news.cgtn.com/news/2024-03-03/AI-deepfake-scams-tric...

https://www.trendmicro.com/vinfo/es/security/news/cybercrime...

https://www.newyorker.com/science/annals-of-artificial-intel...

testacpwoek6• 2 years ago

Making AI-voice videos of Trump and Biden playing Minecraft is one thing, they clearly sound overly robotic and that makes it funny, but being able to perfectly replicate voices is a realistic way is a technology that has very little real-life utility and so, so many real-life downsides.

Literally what is the point of this tech? To eliminate voice actors? I don't buy for a second their supposed use-cases of accessibility or assisting people with disabilities. We're gonna enable people to fake voices realistically off of 15 seconds of for example a voicemail just so media companies can save a few bucks? This is actually madness.

CatWChainsaw• 2 years ago

The only "legitimate" use I can think of is making call center AI agents sound like people. And all the other ones involve getting enough of someone's "voiceprint" to impersonate them to get money or ruin reputations.

Plankaluel• 2 years ago

VoiceCraft, which has been released open source a week or so ago, can do the same with 5-10sec audio and is pretty convincing. It's pretty fun to play around with: https://github.com/jasonppy/VoiceCraft

washadjeffmad• 2 years ago

Thanks. TorToiSe has been feeling a long in the tooth, and this looks a little more digestible.

bijoo• 2 years ago

Submitted multiple times in past few days. Here's a link to the most upvoted one as of now: https://news.ycombinator.com/item?id=39865340 VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild (jasonppy.github.io)

Jayakumark• 2 years ago

Why is this big news, there are open source models that can do this for quite some time , including recently released https://github.com/jasonppy/VoiceCraft

tanepiper• 2 years ago

That scene from Sneakers takes on a whole new level of meaning - we're actually there now...

helsinkiandrew• 2 years ago

"All things are lawful, but not all things are beneficial or advantageous"

St Pauls Letter to the Corinthians 10:23

ChrisArchitect• 2 years ago

[dupe]

Some more discussion on official post: https://news.ycombinator.com/item?id=39866493

OpenAI says it can clone a voice from just 15 seconds of audio

Related Stories