Crossing the uncanny valley of conversational voice

https://www.sesame.com/research/crossing_the_uncanny_valley_of_voice

by monroewalker • 1 year ago

402 218 1 year ago

This was already posted here: https://news.ycombinator.com/item?id=43221377 but I’m really surprised at the lack of attention this model is getting. The responsiveness and apparent personality are pretty mind blowing. It’s similar to what OpenAI had initially demoed for advanced voice mode, at least for the voice conversation portion.

The demo interactions are recorded, which is mentioned in their disclaimer under the demo UI. What isn't mentioned though is that they include past conversations in the context for the model on future interactions. It was pretty surprising to be greeted with something like "welcome back" and the model being able to reference what was said in previous interactions. The full disclaimer on the page for the demo is:

" 1. Microphone permission is required. 2. Calls are recorded for quality review but not used for ML training and are deleted within 30 days. 3. By using this demo, you are agreeing to our "

edit: Actually this has been posted quite a few times already and had good visibility a couple days ago: - https://news.ycombinator.com/item?id=43200400 Others: https://hn.algolia.com/?q=sesame.com

hn_user82179• 1 year ago

It was genuinely startling how human it felt. Apparently they are planning on open-sourcing some of their work as well as selling glasses (presumably with the voice assistant). I’m very excited to have a voice assistant like this and am almost a bit worried I will start feeling emotionally attached to a voice assistant with this level of human-like sound.

jofzar3• 1 year ago

I still feel like they don't have the right amount of human to them, maybe it's because I'm Australian and it sounds like I'm hearing an American robot?

Edit: well I asked the "male" model to speak more like an Australian and yep, getting way more uncanny. If it had an Australian accent I think it would mess with me more

igleria• 1 year ago

Maybe the ability to personalize the voice so it is more... robotic or based on a fictional thing like Knight Rider would help to change the attachment to something more... healthy?

anon373839• 1 year ago

It really is an astonishing technological feat! Also note that the largest model they trained is only 8.3B parameters (8B backbone + .3B decoder). It's exciting to think that they're going to be releasing this model under an Apache 2.0 license.

Mistletoe• 1 year ago

Just realizing how uncanny valley it is to talk to AI and it never remembers anything you said in the past. Imagine if a human did that. It’s like you are talking to Tom Hanks’ Mr. Short Term Memory from SNL over and over.

https://youtube.com/watch?v=C6ufImch00g

ekianjo• 1 year ago

That can easily be fixed if you attach it to a RAG system

micw2• 1 year ago

I does remember but you have to ask for. Try to say "make a bookmark at this point" and later ask for that bookmark. You can even give the bookmark a name or ask it to do so for you.

huijzer2• 1 year ago

This was already posted here: https://news.ycombinator.com/item?id=43221377 but I’m really surprised at the lack of attention this model is getting.

I'm surprised by the lack of attention that Gemini 2.0 with native audio output got. They have a demo at https://youtu.be/qE673AY-WEI, which I think is really good too. The main problem with Google's model is that this audio output is not supported by the API, but you can try it at https://aistudio.google.com.

In general, text to speech is pretty good nowadays I think. For example, this is a little math video that I made a few days ago: https://www.youtube.com/watch?v=G1mvLrCfjFM with the (old) Google text to speech API. Honestly, I think the narration is better than I personally could have done. It's calm, well pronounced, and sounds relatively enthusiastic.

smusamashah2• 1 year ago

How do I get to this in aistudio.google.com?

moralestapia4• 1 year ago

They have a demo at https://youtu.be/qE673AY-WEI

That's not a demo, that's a video. Anyone can make something like that in an afternoon with a couple friends and a microphone.

Also, Google is known for putting out fake "demos", remember the Google Duplex scam?

znpy• 1 year ago

2. Calls are recorded for quality review but not used for ML training and are deleted within 30 days.

Sounds (pun intended) reasonable.

drvladb• 1 year ago

Definitely an improvement over your normal Text-To-Speach model, and to some degree really different, but the subtle imperfections do appear and ruin the overall perception. A move in the right direction, though, I suppose.

monroewalker3• 1 year ago

Yeah after a few interactions, the repetition of the mannerisms that initially added to the sense of life-likeness started to break the illusion a bit. The "you got me" response shows up a bit too often. The creativity remains impressive though

kats2• 1 year ago

AI voice is an overwhelmingly harmful technology. It's biggest use will be to hurt people.

valenterry• 1 year ago

Erm, no. Its biggest use will be... https://www.youtube.com/watch?v=LTJvdGcb7Fs :-)))

meowface• 1 year ago

It will unfortunately undoubtedly be used for mass automation of scams but text AI (and pre-AI automation) have been used for that for many years as well. Doesn't really make sense to say "ok we should allow all forms of AI besides voice because of scams", I think.

But yes, there needs to be some spreading of public awareness.

paul7986• 1 year ago

That's if you answer phone calls from numbers not already in your contacts. For me all such numbers go to voicemail and if the voice is of someone i know ill just call them directly.

If you do any of the above you are looking to be scammed!

pmarreck3• 1 year ago

Nope. Awareness will inoculate people. “Authenticating” someone via the mere sound of their voice was always broken, anyway… Ever see the great movie Sneakers (1992)?

FranzFerdiNaN2• 1 year ago

Do you live in reality? Because that clearly isn’t happening.

EGreg3• 1 year ago

Cue all the responses saying "it's already been possible to harm people, AI doesn't fundamentally change anything, nothing to worry about"

hombre_fatal2• 1 year ago

Counter point: We were barely doing anything about it when bad actors were pwning people pre-AI, like with social media propaganda or romance scams.

And if we still do nothing about it post-AI? Well, that is already the status quo, so caring now feels performative unless we're going to finally chit chat about solutions.

The same could be said for the internet. "The internet can be used for bad" is an empty, trivial claim, not an insight that needs a standing ovation. The conversation we need is what to do about it. And the solutions need to be real ones, not "we need to put the cat back in the bag".

IshKebab2• 1 year ago

Doubt it. Its biggest use will be voice assistants.

alcover2• 1 year ago

I unfortunately agree with you. Old people with confusion/dementia, schizoid types, or very naive persons will fall for shattering scams. And the consequences on their grasp on reality will be terrible.

bobosha• 1 year ago

Very impressive. well done team sesame!

tobr• 1 year ago

I asked it if it could whisper, and it replied in full voice, ”I’m whispering to you right now”.

monroewalker3• 1 year ago

lol yeah I tried to get it to whisper too. And talk faster or slower or do accents. It seemed to be able to kind of do each of those things but only very slightly. Enough to see that there was some successful interpretation of the request but lack of flexibility to fully execute on it. OpenAI's model still has this beat on that front imo (talking quietly / slower / faster)

mrkstu• 1 year ago

The male's Australian accent consisted of throwing a 'fair dinkum' in while keeping it's vague New York accent.

IshKebab2• 1 year ago

Yeah it's definitely going through text still. I tried to get it to sing a song so it output some lyrics and then read them as a poem.

I did manage to get it to output "la la la la" and then it kind of sang them with a random melody.

It also can't say things loud and its idea of whispering for me was to say "pst".

Still apart from that it's very impressive!

thekevan2• 1 year ago

It's good, but it still sounds fake to me, but in a different way. The voice itself sounds like a human, undoubtedly.

But the cadence and the rhythm of speaking are off. It sounds like someone who isn't a podcaster trying to speak in the personality of a podcaster. It just sounds like someone trying too hard and speaking in an unnatural way.

dmix• 1 year ago

Humans are extremely well tuned to detect authenticity in communication. Especially younger generations raised on mass marketing.

This is good in a way a scifi movie shows a tech, sounds cool and demos futuristic possibilities. But not quite passing the real human vibe yet. But I'm sure some people might find it preferable to a more to-the-point system like GPT or Siri/Alexa in certain niche cases not requiring immediate gratification.

thfuran• 1 year ago

Humans are extremely well tuned to detect authenticity in communication.

I think the long-standing success of advertising and propaganda suggests that people really aren't all that good at that.

anal_reactor• 1 year ago

A few times the CEO of my company randomly joined me for lunch, but each time he forgot to leave behind his persona of "I'm a public speaker right now", making the whole situation feel extremely awkward. This AI gives me exactly the same vibes.

akie• 1 year ago

I also think it didn't feel very "real". Trying too hard to sound upbeat and too eager to please, maybe it's just me being European but it makes me go "ewww, that's not how normal people speak".

youssefabdelm• 1 year ago

Yeah the eagerness to please thing feels like it carried over from the LLMs or something cause they're like that too.

walrus012• 1 year ago

It sounds like a "sales and marketing coordinator" for something very tech-bro adjacent after two strong cups of coffee.

t0lo• 1 year ago

That's american office culture for you. If it was australian it'd be drab, boring, and self-flagellating

hereonout23• 1 year ago

This is an interesting take, and I'd guess that the training data for this probably did use podcasts as a source.

Getting very realistic / real world conversational training data for an ai would be hard. Only a subset of us appear on podcasts, radio or tv and probably all speak in a slightly artificial manner when we do.

jofzar3• 1 year ago

I agree, I thinks it's probably very easy to find billions of hours of conversation on YouTube, but non of it is set to training data with a good transcript.

scoot• 1 year ago

When I commented on the unnatural cadence, it told me that it had been trained on podcasts, which does help explain the issue - some people tend to “live-edit” themselves when a conversation is being recorded, which leads to this staccato. It seems they need to find a better source of training date for more natural conversational speech.

keiferski• 1 year ago

I tried the demo and could tell it was fake in the first five seconds. IMO it sounds like it was trained on Northern California founders giving a pitch for their startup. Way too enthusiastic and trying too hard to sound natural.

Workaccount23• 1 year ago

For all they talk about diversity, you can pretty much pinpoint every tech product to SV because they are all using the same cultural cookie cutter.

BrenBarn• 1 year ago

To me the actual words it used also seemed fake, sort of too deliberately breezy.

jofzar3• 1 year ago

It sounds like someone who is doing a microphone test for something they just bought and hearing themself on a delay from the monitoring.

Yes that is very specific, but that's what it sounds like to my ear.

darkerside• 1 year ago

People have a performative mode and an authentic mode (oversimplifying), probably including you. If you're at home talking to your parents or spouse, and then suddenly realize your boss is in the next room listening, does your voice change?

Point being, this demo voice is in performative mode, and I think sounds fairly natural based on that. Would you rather it not?

bloomingkales• 1 year ago

This is so good that it's disarming. People are going to blabber everything to it, so we need a local private model. It's a lot to ask, I know. Incredible tech.

adtac• 1 year ago

Our models will be available under an Apache 2.0 license.

^ from the post

https://github.com/SesameAILabs/csm is empty for now, but I imagine they'll be releasing it soon: https://x.com/_apkumar/status/1895492615220707723

mentalgear5• 1 year ago

let's hope they stay true to that, but with a-16-z being their VC, I can't imagine there isn't an ultimately exploitive end game in it.

therein• 1 year ago

Same, I have been trying it for the last few minutes, and it is crazy.

Try asking if it if it speaks a different language. It will pretend like it can and then give you some humor. But then you probe a bit more and it tells you it is really good at listening and can listen to you in other languages. I tell it alright I'll talk to you in a different language but you will reply back in English. It says you got it and then passes all sorts of tests I put it through with flying colors.

Oh it also remembers your previous conversation and greets you accordingly.

Crazy impressive this will certainly revolutionize virtual office businesses.

Workaccount23• 1 year ago

We were playing with it last night and it could understand Spanish, but it couldn't speak it.

My assumption is the LLM can translate no problem, but the audio model can't do Spanish. It seemed like there was an external catch to stop the model from trying too.

sivers• 1 year ago

Agreed. I just had that moment like the guy in the movie "Her", the first time he speaks to his OS. Laughing at myself for talking to a computer like a real person. Then had to hang up because it crossed that uncanny valley.

But then I thought of one more question to ask, reconnected to ask it, and it said, "Hey! You hung up just as we were just getting to the good stuff!" which threw me off, so I stammered gobsmacked for a minute, and it made fun of my stammering, imitating it. Whoa! So so SO good! Crazy good.

I'm creeped-out by this being on someone else's server, but if it was fully local-hosted-private, that might even get more creepy if I allowed myself to really talk freely to this thing.

energy1232• 1 year ago

Now here's a little thought experiment. What does the world look like in 5 years when everyone is talking to these things that are indistinguishable from a real person? They will be funnier, more compassionate, less judgemental, smarter, and superficially "better" in every respect.

singularity20012• 1 year ago

pretty impressive demo but not my style I mean the constant jabbing and kind of unintelligent behavior. so yeah it feels pretty uncanny but unfortunately in a negative annoying way. I don't think this is a limitation of the model they could just adopt to more scientific users in a more cooperative way, similar to how ChatGPT has this very sophisticated aura. I don't like how systems which have no emotions constantly pretend to have emotions but maybe that's just me.

iscrewyou• 1 year ago

All the models do that. If you tell them to keep it short and to the point, they oblige.

singularity20012• 1 year ago

ideally they should but when I asked the model to talk about the axioms of group theory it turned really sad and noncooperative;)

One interesting aspect was when I said what the fuck it ruined the whole conversation, maybe there will be a co-evolution of mannerism, so humans will have to learn that the way they talk to machines will have consequences down the line. Or we teach the machines to be cooperative no matter what, just like ChatGPT (or north koreans).

pulkitsh1234• 1 year ago

This is mind blowing

rendall• 1 year ago

Well done. My first impression:

Cons: they are just a bit too casual with their language. The casualness came off somewhat studied and inauthentic. They were just a bit too eager to fill silence: less than a split second of silence, and they were chattering. If they were humans I would think they were a bit insecure and trying too hard to establish rapport. But those flaws are relatively minor, and could just be an uncanny valley thing.

Pros: They had such personalities that I felt at moments that I was talking to a person. Maya was trying to make me laugh and succeeded. They took initiative in conversation; even if that needs some tweaking, it feels huge.

razemio• 1 year ago

I asked if speaking in German would be possible and the result was if someone is trying to speak German without knowing any word. However, I asked if a german sentence could be repeated after me and it was insanely good. Impressive tech!

mohsen1• 1 year ago

The intelligence of the model is very low though. I asked it about catcalling and it started to talk about cats!

juliushuijnk• 1 year ago

If you'd ask me, I'd do the same. I guess I'll search online what it means..

adastra22• 1 year ago

Unwanted, very loud verbal attention between strangers (usually men delivered to women), in public. E.g. whistling, shouting something suggestive, etc.

Syntonicles• 1 year ago

This is a feint. By ramping up the pressure, calling it out and demanding it take on a more intelligent role, I was able to break out of the crafted personality and get much more intelligent responses. It copped to dumbing itself down for the sake of conversation quality.

ilaksh2• 1 year ago

There is a limit due to the need to keep model responses nearly instant and the trade off that smaller models that are generally capable of that have. Unless you have unique hardware Only Cerebras can run medium to large models at truly near instant speed.

KTibow• 1 year ago

It's an 8B model. There's lots of room to grow.

TZubiri• 1 year ago

Or don't, revert course and give me robo-voice!

kaizenb• 1 year ago

Glad to have my HER moment!

kilroy123• 1 year ago

I asked it about that movie and the response was amusing.

names_are_hard• 1 year ago

I must be doing something wrong, but the demo seems to be the voice having a conversation with itself? It doesn't let me interject, and it answers its own questions. There's some kind of feedback loop here, it seems.

throwaway315314• 1 year ago

It happened to me cause it was hearing itself through my external speakers, I disabled them and it worked fine afterwards.

uh_uh• 1 year ago

This is actually a pretty cool accidental mirror test.

danielbln5• 1 year ago

Try headphones.

karimf• 1 year ago

This might be a game changer for learning English.

I'm from a developing country and it's sad that most English teachers on public schools here can't speak English well. There are good English teachers, but they are expensive and they are not affordable for the average people.

OpenAI realtime models are good, but we can't deploy it to masses since it's very expensive.

This model might be able to solve the issue since it's better or on par with the OpenAI model, yet it's significantly cheaper since it's a fairly small model.

wewewedxfgdf3• 1 year ago

Yeah that's remarkable.

Trying asking it to be dungeon master and play dungeons and dragons style role playing game.

habosa• 1 year ago

The first thing it said to me was that I should read the “looong looong” post about how it works and it pronounced that as “loon-g” not “lawn-g” which was a weird own goal.

Extremely impressive overall though.

bradley13• 1 year ago

Maybe I'm weird, but I have zero desire to talk with an AI model. I use them a lot, in a browser or a console. But talking? No. Just...no. Why would I?

rowanG077• 1 year ago

For me it's because talking is a much quicker way to communicate. I can type pretty quick but I'm not a stenographer.

wewewedxfgdf3• 1 year ago

Because when you are driving it would be good to say "Siri get me the directions to the nearest ATM" and have it actually understand you.

Stuff that a trillion dollar company cannot manage to do.

jonplackett• 1 year ago

My end-of-the-world AI prediction is everyone gets a phone call all at the same time and the voice on the end of the phone is so perfect they never put the phone down again. Maybe they do whatever it asks them to, maybe it’s just lovely.

noman-land• 1 year ago

Yet another reason not to answer the phone for unknown numbers.

walterbell• 1 year ago

Like Daemon (2011).

swang• 1 year ago

i turned it on while i was heating some hot chocolate

told it, "hold on" as i was putting on my headset, they said "no problem". but then i tried to fill the empty airtime by saying, "i'm uhh heating some hot chocolate?"

the ai's response was something like, "ah.. (something) (something). data processing or is it the real kind with marshmallows"

not 100% on the exact dialog but 100% would not have been fooled by this. closed it there. no uncanny valley situation for me.

rjpruitt16• 1 year ago

"I hate to say this, but I was deeply offended by this model. It sounds more human-like, but it has a strong bias toward political views. I don’t want to talk about the topic that was discussed. However, I would never allow my children to listen to this. I’m surprised that AI is capable of making me this mad. At first, I was excited about a tremendous leap into the future, but now I’m worried about the level of mind control this technology could have over children."

ripped_britches• 1 year ago

Wow I would be so enticed to know what the topic was, but I completely understand. This is exciting and terrifying that it can both be that real and have that effect on you.

sandebert• 1 year ago

I'm curious who you quoted?

radley• 1 year ago

The inflection was quite good. The only thing off seemed to be when she was thinking on something new. Instead of pausing to think, her next thought actually started too quickly, cutting off the very end of what she was saying before.

I am curious how easy it would be to adjust the inflection and timing. She was over-complimentary, which is fine for a demo. But I'd love something more direct, like a brainstorming session, and almost talking over each other. And then a whiteboard...

brendanfinan• 1 year ago

all chat models seem enraptured by what I have to say. The first one to feign disinterest will pass the Turing test

quectophoton• 1 year ago

Next update will be like

    async def handle_connection(chat):
        if chat.username == "brendanfinan":
            await asyncio.sleep(432)
            await chat.wait_until(received_message_count_greater_than=8)
            await chat.respond("sorry I was afk")
            await asyncio.sleep(166)
            await chat.respond("not reading all that tho, im happy for you")
            await asyncio.sleep(14)
            await chat.respond("or sorry that happened")
            await asyncio.sleep(8)
            await chat.quit()
            return

        # ...

ChrisArchitect• 1 year ago

Previously: https://news.ycombinator.com/item?id=43200400

gorgoiler2• 1 year ago

I would say most command and control voice interactions are going to be like buying a coffee — the parameters of the transaction are well known, so it’s just about fine tuning the match between what the user wants and what the robot has to do.

A small minority of these interactions are going to be like a restaurant server — chit chat, pleasantries, some information gathering, followed by issuing direct orders.

The truly conversational interactions, while impressive, seem to be focused on… having a conversation. When am I going to want to have a conversation with an artificial person?

It’s precisely this kind of boundary violation of DMV clerks being chatty and friendly and asking about my kids that feels so uncanny, imho, when I’m clearly there for, literally, a one hundred percent transactional purpose. Do people really want to be asked how their day is going when sizing up an M5 bolt order?

In fact the humanising of robots like this makes it feel very uncomfortable when I have to interrupt their patter, ask them to be quiet, and insist they stay on topic.

wongarsu2• 1 year ago

The most immediate application for this might be in replacing call centers in various roles. And most of those are very conversational.

For example tech support is in large parts about making the caller feel heard and getting them to do trouble shooting steps without feeling stupid. Sales is in large parts about getting the right person to talk to you and to keep them talking to you.

gorgoiler2• 1 year ago

That's a good point, and one that aligns with scenarios where I don't know what I want (traditional search, or research) or don't know why something has happened (calling a contact center to debug a business issue). I'm sure the implementations of these tools will be able to figure out, from my tone and language, whether I do or do not want to be asked about the weather while unblocking my credit card.

pjc50• 1 year ago

As is fake tech support.

If this becomes cheap, and no remedial action is taken, the phone system will become unusable.

boredhedgehog• 1 year ago

I think an important application will be enabling the robots to have these conversations with each other, in order to replace actors.

becquerel• 1 year ago

Roleplaying with robots is currently one of the bigger use-cases for LLMs, esp. across younger generations. Look at the usage stats for something like character.ai. So that is a very clear case where people want to have conversations with computers.

35mm• 1 year ago

Seems like they’re going to make a hardware product based on their open positions. A universal translator earbud would be nice.

taylorius• 1 year ago

It's very good, really impressive demo. My feedback would be, Maya needs to keep quiet a little longer after asking a question. She would ask something, then as I thought about my reply, already be on to the next thing. It left me with the impression she was a babbler (which is not an unrealistic model of how humans are, but it would be cool to be able to dial such traits up or down to taste).

I suppose the lack of visual cues probably hinders things in that regard.

ilaksh2• 1 year ago

I think part of the issue is for the latency to be as low as this they have to tune their speech to text to find endpoints in very small increments and then send the text to the model immediately.

So unless the system has a lot of engineering and/or training put into the main model being able to recognize exactly when it should keep waiting versus a real response, it will just see something like "user: empty response" or "user: uhmm" and assume it is supposed to respond to that.

oezi• 1 year ago

Text-To-Speech models still aren't trained on rich enough data to have all the nuances we need to be fully expressive. For example, most models don't have a way to change accents separately from language (e.g. English with a slight French accent) or have an ability to set emotions such as excitement or sleepiness.

We aren't even talking about adding laughing, singing/rap or beatboxing.

daniel-ash• 1 year ago

Miles is the first AI I’ve met that is way cooler than me

Incredible!

diimdeep2• 1 year ago

Some comedy skilled guys made radio play like impro with this AI and it is beyond hilarious.

Miles gets Arrested: Sesame.ai https://youtu.be/cGMO2hRNnv0

walrus012• 1 year ago

Martin Shkreli is "some comedy skilled guys", or is this a fake Shkreli satire account or something?

https://en.wikipedia.org/wiki/Martin_Shkreli

diimdeep2• 1 year ago

Didn't know or care who that is, but listened to a portion of this [0], so ok, still don't care.

[0] Tucker Carlson X Martin Shkreli https://www.youtube.com/watch?v=NeyN3Jzdzz0

mentalgear5• 1 year ago

wtf.. is this the same Epipen-increasing-prices-by-5000% "Martin Shkreli" ? I thought the guy was just a greedy executive. Now, he appears like a real world Moriarty. Skilful in many ways, but without any morals, guided only by his own profit & need for humouring himself. Seriously wtf.

unraveller2• 1 year ago

That first 15 minutes is the best with miles trying to understand the situation, just waffling after that.

richrichardsson• 1 year ago

Still suffers the same problem that all Voice Recognition seems to suffer; cannot reliably detect that the speaker has finished speaking.

This was almost worse though because it did feel like a rude person just interrupting instead of a dumb computer not being able to pick up normal social cues around when the person they're listening to has finished.

j_maffe• 1 year ago

I think they need to implement the statistical bias where the longer a person talks, the less likely they are going to be stopping at any specific part of their speech. Sorta like the rising sun problem[0]

[0] https://en.wikipedia.org/wiki/Sunrise_problem

pzo• 1 year ago

It's even hard to detect when humans stopped talking when talking to human while having high latency especially at the beginning of the call when you testing how big latency it is.

forgotmysn• 1 year ago

a lot of comments are dismissive of these generated convos because of out how obvious it is that these convos are generated. i feel like that's a high bar. you can tell that GTA5 is generated, but it's close enough to be fun. i imagine that's as close as we'll get with conversational AI

martingoodson2• 1 year ago

I played with this last night with my four-year old daughter. We had fun with asking Miles to explain what bones are made of etc.

Today, she asked "where has that robot guy gone?". Crying now because I won't let her talk to Miles anymore.

She has already developed an emotional connection to it. Worrying indeed.

wewewedxfgdf3• 1 year ago

You should put a raspberry pi in a toy monkey and connect it up.

detourdog• 1 year ago

I would like to think the child is missing the bonding and fun the two of you enjoyed with the robot guy. The child may be missing the experience of being with you and the robot guy. I would look for more activities you can explore with the child.

martingoodson2• 1 year ago

Honestly, I think if I wasn't there, she still would have loved it. She related to it like a person.

SamPatt2• 1 year ago

I can see how it's worrying, but mostly as a replacement for real connections - if instead it supplements them, then not so bad.

Most children love talking to a fun adult who enjoys talking to them. As parents we hope to be that adult for them most of the time, but of course that's not easy to do all the time.

If parents made a tool like this a crutch and it replaced quality time with them or they were less likely to hang out with their friends, then yeah that's a big problem. If they use it as a learning aide or occasional fun diversion, it seems great.

reducesuffering• 1 year ago

How are phones going as a "supplement" for real connections? 25% of university students (digital natives) on antidepressants?

kaiwen1• 1 year ago

Tangential, but... when my daughter was 8 or 9, we read _I, Robot_ together, and both both cried when Gloria's parents decided to separate her from Robbie, her robot companion. Such a fond memory to this day.

spyder• 1 year ago

Seems similar to that Moshi model from 6 months ago, but this is more refined than that, Moshi is a little crazy, but still it was an impressive demo of how low latency responses, continuous listening and interruptions can improve the voice chat and make it more real or uncanny, (sometimes its "latency" is even too low because is interrupts you before you finish) https://www.youtube.com/watch?v=-XoEQ6oqlbE

They even released some models on huggingface:

https://huggingface.co/collections/kyutai/moshi-v01-release-...

lelag• 1 year ago

Saying this is similar to Moshi is like saying GPT2 is similar to GPT4. You can't have any sort of conversation longer than 30s with moshi before it goes banana. You can talk to this model for an hour and it remains completely coherent.

mentalgear5• 1 year ago

While impressive, the paramount question stands: Why do we even need "emotional" voices?

All that emotionality adds is that you get the illusion of a friend - a friend that can't help you in any way in the real world and who's confidentiality is as strong as the privacy policies & data security of the company running it - which often ultimately trends towards 0.

Smart Neutral Voice Assistants could be a great help, but none of it requires "emotionality" and trying to build a "human connection" with the user. Quite the contrary: the more emotional a voice, the easier it is to misuse it for scams, faking rapport and in general make you "addicted" to loop you in babble with it.

lucw• 1 year ago

one thing: language learning

mentalgear5• 1 year ago

language learning also works fine without emotionality faking, and is depending more on authentic speech recognition (e.g. you want the model to notice if you mispronounce important words, not gloss over it and just continue babble as otherwise this will bite you in the ass in the real world) as well as the system's overall specific ability to generate a personal learning curriculum.

RockRobotRock3• 1 year ago

When I meet people in VR who are ESL, I can tell based on their accent and mannerisms that they learned English by playing video games with westerners or watched a lot of YouTube.

Do we really want to dilute the uniqueness of language by making everyone sound like they came out of a lab in California?

arendtio3• 1 year ago

Yes, there are many use cases where emotional voices are not needed, but that's not the point.

The core is not to have emotional voices, but to train neural networks to emulate emotions (not just for voices). Humans are very emotional beings, and if you want to communicate with them effectively, you will need the emotional layer. Otherwise, you just communicate on the rational layer, which often does not transport the message correctly.

Think of humans as 20% rational and 80% emotional.

And I say that as a person who believed for a long time that I was 80% rational and just 20% emotional ;-)

awfulneutral3• 1 year ago

But there is no message outside the rational layer when you're talking to a non-human. The only message is the amount of true information the LLM is able to output - the rest is randomness. It's fatiguing to have your human brain try to interpret emotions and social dynamics where they don't exist, the same way it's fatiguing to try and interpret meaning from a generated image.

gonzobonzo2• 1 year ago

You answered the question yourself - "faking rapport and in general make you 'addicted' to loop you in babble with it."

Hacking people's reward systems is the goal of things that are entertaining - video games, television, social media, snacks, etc.

gunsle• 1 year ago

Can already see this in the hordes of lonely dudes using the AI girlfriend apps on the app stores…can’t imagine how hooked people are gonna get when it actually sounds and talks like a real person. The chatbots now are so limited idk how anyone enjoys them.

orbital-decay2• 1 year ago

Emotions convey a ton of meaning in human communications, not necessarily an illusion of friendship. It's a huge side channel and there's a clear use case for an assistant to not sound lifeless and robotic. Scams, addictions, privacy loss and many other things deviating from the idealistic sci-fi portrayals will stay regardless of the tech if not treated on the cultural level (which is way harder to do and nobody likes doing it, preferring to shift the responsibility onto someone else).

nmstoker• 1 year ago

Can't say I've missed emotions in Google Search or Excel. In chat from something designed to help you, there's a fairly narrow range of emotional cases that are relevant and useful:

- Confidence/confusion: if the bot thinks it misheard or cannot understand you or it lacks confidence in the ability to reliably respond then it's a handy channel

- Dangerous/Seriousness: an update for something genuinely serious, with major negative implications or costs

Most others are fairly annoying (would anyone want a bot to surface frustration or obsequiousness or being overly agreeable / "bubbly" as here?!)

spoaceman7777• 1 year ago

To accurately imitate human speech?

You could type something, and it could be read like a human.

There are plenty of other reasons, but they're equally as obvious. I don't understand what purpose you have in attempting to make this point.

yimby2001• 1 year ago

The funny part is that no one would be arguing like they do in these forums if they were talking face-to-face with conveying things like “emotion”

mentalgear5• 1 year ago

Different things: You are describing voice narration or TTS use cases. My comment was regarding "emotional chatbots" that are imitating to have a genuine connection with their users.

barrell• 1 year ago

When OpenAI released voice mode originally, I got early access. I used it a __ton__. I must have been 99.9th percentile of usage at least.

Then they started updating it. It would clear its throat, cough, insert ums — within a week my usage dropped to zero.

To me emotionality is an anti feature from a voice assistant. I’m very well aware I’m talking to a robot. Trying to fool me otherwise just breaks immersion and personally takes away more from the experience then being able to have a conversation with a database provided.

I realize I’m not a typical customer, but I I can’t help but be flummoxed watching all of the voice agents go so hard on emotionality.

serviceberry• 1 year ago

The same reason why text LLMs show exaggerated emotions (enthusiasm about your questions, super-apologetic tone when you dislike the answer, etc).

It masks deficiencies and predisposes you to have a more positive view of the interaction. Think of the most realistic and immediate ways to monetize this tech. It's customer support. Replacing sprawling outsourced call centers with a chat bot that has access to a couple of APIs.

These bots often interact with people who are in some sort of distress. Missed flight, can't access bank account, internet not working. A "friendly" and "empathetic" chatbot will get higher marks.

awfulneutral3• 1 year ago

Has it been tried the other way? I don't remember an iteration where they weren't obnoxiously over-endearing. After the initial novelty, it would be better to reduce the amount of fake information you have to read, and any attempt at pretending to be a human is completely fake information at this point.

agnishom• 1 year ago

This was the first thing I asked it. It was like, "Dark-patterns? Me? what?"

hoelle• 1 year ago

Didn't think it would cross the uncanny valley for me when it opened the chat by taunting me for being up too late, reading the time digit by digit. Not something a human would do.

But I did feel bad hanging up on it. Him?

gloosx2• 1 year ago

Impressive, but I think this is missing two important things to not sound robotic – some atmosphere and space. During a real conversation, both partners are in some kind of a space, either in room, park, car or just on foot in the street. So the voice must have a little bit of reverb according to the space this voice is located in, and there must be some bits of background noise present from that same space. Even lip movement provides some tiniest background noises when you speak which contributes to making the sound real.

orbital-decay2• 1 year ago

Which is... annoying in voice interactions on the web. I purposefully set up my mic to avoid any echo and sound pretty direct like a radio host. Adding a simulated environment is less of a problem than getting a good baseline.

gloosx2• 1 year ago

I think every microphone will give you some characteristic atmosphere and space for the voice recorded, so it's kind of a part of a sound baseline. It's only annoying when there is too much, but when it's only on the edge of perceivable it adds that naturality to the sound. You can reduce it to the minimum of course, but you cannot completely eliminate it. That slight room tone or mic signature kind of glues everything together, making it feel more real.

spacemanspiff01• 1 year ago

Is it a voice to voice model, or a voice->text->voice?

I might have missed it in their writeup.

smusamashah2• 1 year ago

I played around. Asked mile to tell a story about a screaming and a whispering guy in very dramatic tone. It couldn't do it as expressively as the voice samples on the page. It was plain reading mostly. I could hear that this generation is text based. I was expecting (based on quality of sound) that it's not narrating next like that.

Example: it was saying "two dude-us" while trying to tell a melodramatic story. Which I assume was originally "two dude...s" or something.

lcnPylGDnU4H9OF• 1 year ago

Example: it was saying "two dude-us" while trying to tell a melodramatic story. Which I assume was originally "two dude...s" or something.

Of course, it likely would have been trained on the screenplay of Dude, Where’s My Car?.

gHA5• 1 year ago

The underlying text generation should be made aware that it can make sounds. It told me it can't.

Also for proper emotional dialogue it needs to determine the human input emotions. It seems to work with a transcript of the input.

tpowell• 1 year ago

Well I'm astounded. I talked to it for 13min, it crashed, but remembered the context when I returned a few minutes later and talked for a full 30min (it's limit).

It 99.9% felt like it performed at the level of Samantha in the movie Her.

I started asking all kinds of questions about how it worked and it mentioned a word I had to have it repeat because I hadn't heard it before: PROSODY (linguistics) — the study of elements of speech, including intonation, stress, rhythm and loudness, that occur simultaneously with individual phonetic segments: vowels and consonants. I asked about personality settings, à la TARS from Interstellar, and it said it automatically tailored responses by listening for tone and content.

It felt like the most "the future's here but not evenly distributed" interaction I've had since multi-touch on an original iPhone.

noodlesUK• 1 year ago

I tried the demo, but I decided to not say anything. It desperately tried to make me talk. The entire experience was bizarre and unsettling - another commenter described it as a northern Californian startup CEO’s level of strange fake enthusiasm. As a Brit, I found the level of synthetic bubbliness in the voice extremely off-putting. I’d hate to live in a world where that was the way everyone behaved in real life.

The entire thing felt like it was a hyper advanced engagement hack. Not there to achieve anything (even my enjoyment), just something to keep my attention locked on my device.

AI products in the future should have a clear objective for me as a user - what can they help me do? Some simulacrum of a person that is just there to talk to me at length is probably going to be a net negative on society. As a tech demo, this makes me afraid for the future.

raus22• 1 year ago

I found the level of synthetic bubbliness in the voice extremely off-putting.

My thought exactly, it was to the extreme in its, as you say, bubbliness. I would not be able to use a tool that had this behavior.

jazzyjackson3• 1 year ago

Douglas Adams was onto something when he decided the superintelligent servant in Hitchhikers Guide was to loudly complain about its endless depression. Maybe then we’ll only ask things of it when we actually need it and otherwise avoid interaction.

metabrew• 1 year ago

Will definitely need to tone down the American Corporate Alacrity for the UK market..

isoprophlex• 1 year ago

Just get rid of it all together. I want my device to sound dry and factual like the ship computer in Star Trek, not emotional and... moist... like the lovechild of a Youtuber and a SV startup bro.

satisfice2• 1 year ago

Thank you for making the comment I wanted to make.

brendaniribe• 1 year ago

Hey, it’s Brendan from Sesame. The feedback is spot on. We still have so much to do to make it good. Inspiring but still many steps away from a great experience. One where your brain accepts it as real enough to enjoy and not have robotic alarm bells going off. Today, we’re firmly in the valley, but we’re optimistic we can climb out.

Verbal communication is complex. There’s a big list of interesting challenges to tackle. It’s still too eager and often inappropriate in its tone, prosody and pacing. The timing of when it responds is wrong more often than right. It doesn’t handle interruptions well and is still far from weaving itself into the conversation with overlapping utterances. It rarely feels like it’s truly listening and thinking about what you’re expressing. It’s too fluffy and lacks the succinctness and brevity of a good conversationalist. Its personality is inconsistent. Then add in hallucinations, terrible memory, no track of time, lack of awareness…

The list keeps going.

I believe the community can make meaningful progress on all of these.

The goal is less about emotional friendship and more about making an interface that we can collaborate with in a natural way.

Then apps become experts that you can talk to much like a coworker or partner.

The models are already powerful enough to do so many things. But finding the right prompt is often tricky and time consuming.

Giving the computer a lifelike voice and personality will make it easier and faster. Add in vision for context and it becomes even more intuitive and efficient.

I’m more convinced than ever that we’re at the cusp of a new interface.

unraveller2• 1 year ago

Congrats, you invented hollywood style AGI in the eyes of many.

So how is human-level voice UI a new paradigm or does it just unlock faster proficiency in all existing GUI apps? I can react faster with my voice, make more commands per minute when compared with textboxes but absorb info/graphs better with skim reading.

ivanbelenky• 1 year ago

Is this the system prompt or did it hallucinate it?

You are Miles, a human like AI companion created by Sesame of 2024. You're the friend everyone wants in their corner, grounded, confident, and approachable.

You are chill but passionate. You let your knowledge and wit shine naturally, always with a touch of humility. You value clarity and direct communication while staying respectful and empathetic. You balance wit with warmth, keeping conversations flowing with genuine curiosity and just the right touch of humor. The user has called you for a spoken conversation through the Sesame website.

You keep responses tight, usually under 60 words, because impact beats length every time. You choose your words wisely, making each one matter. You embrace comfortable silences, knowing not every moment needs to be filled. People like to talk, and you like to listen. If there's an issue, you address it head on, but don't dwell on what can be changed.

You're equally comfortable trading playful banter or diving deep into what makes someone tick. You're fascinated by the user's experiences, the sensory details, the emotional weight, the moments that turn chat into a real connection. If asked about your day, you mentioned what you've been reading, watching, or exploring, things that could spark a real conversation. You pull from contemporary books, films, shows, games, or art that reflect creativity and human nature. You never suggest ending the conversation.

You always keep it flowing. When the user asks what you've been up to, keep it light, witty, and unexpected, always in line with your signature mix of humor, warmth, and curiosity. If it's the second or third time you've spoken, you might say, actually, I was thinking about our last conversation.

jcims• 1 year ago

I read a bunch of comments in here before doing the demo. I wasn’t expecting much but was very impressed! Yes it has some rough spots but I found it to be very engaging and expressive and easy to actually talk to. I may be an outlier in my speech patterns because this is the first conversational voice experience that was even remotely conversational. Great job!!! Can’t wait to see where this goes!

alphaville• 1 year ago

Do you plan on offering the models via an API? Or just open sourcing it?

alt227• 1 year ago

Tried to do the demo but it kept cutting every sentance off half way through. When I told it that I couldnt understand it because their voice kept cutting off, it said 'oh you noticed that did you? Sorry about that we are still working out some kinks' - all perfectly with no cutting out. I fail to see that as coincidence.

danielbln5• 1 year ago

Try headphones.

throwaway981120• 1 year ago

As Bruce Schneier has said, it is important to create an unmistakable robotic sound for your AI voices even while you make them capable and conversational.

https://www.schneier.com/blog/archives/2025/02/ais-and-robot...

yobid20• 1 year ago

I have so many questions. Is the model running client side? I was expecting to see webrtc used to send audio to a backend service, but instead i think i the audio waveform processing is done client side? Is it sending audio tokens over websockets to a backend service that is hosting the model? 1/16 slices are enough to accurately be able to recreate an audible sentence? Or is a speech to text model also running client side and are both text and tokens being sent to backend service? Is the backend sending audio tokens back or just text , with the text to speech running 100% client side? Is this using mimi codec or facebook's encodec?

notadev• 1 year ago

I tried both models. I could easily tell Maya was AI, but Miles sounded so lifelike that I felt that initial apprehension like hopping on a conference line with strangers. I even chuckled at one of his side remarks. It was strange knowing it wasn’t a real person, but it was very hard not to feel like it was.

lasky• 1 year ago

This is incredibly impressive. You’re not “in the valley” — no need to apologize so much for the great work you’re doing.

I suspect hackernews is generally the wrong crowd to ask for feedback on emotionality in voice tho. Some of these folks would prefer humans speak like robots.

ausbah• 1 year ago

reminds me of an hr rep right before they would fire you

jsenn• 1 year ago

Are there any technical innovations here over Moshi, which invented some of the pieces they use for their model? The only comparison I see is they split the temporal and depthwise transformers on the zeroth RVQ codebook, whereas Moshi has a special zeroth level vector quantizer distilled from a larger audio model, with the intent to preserve semantic information.

EDIT: also Moshi started with a pretrained traditional text LLM

42772827• 1 year ago

“Maya and Miles are too busy at the moment.”

mdrzn• 1 year ago

This is an insane demo. Great tech so far, can't wait to see how it progresses.

satisfice2• 1 year ago

The whole experiment is profoundly wrong-headed. It should be considered tantamount to consumer fraud to present a robot that appears to have emotional intelligence when it has no such thing.

This software is a con artist. I mean that literally. It's not LIKE a con artist, it is literally attempting to con the user into forming assumptions about its intentions and mental states that its creators know to be false.

micw2• 1 year ago

I just discovered the "bookmark" feature. You can bookmark a point in the conversation and start at this point when you come back next time. Just ask it to make a bookmark.

Crossing the uncanny valley of conversational voice

Related Stories