GPT-4o

https://openai.com/index/hello-gpt-4o/

by Lealen • 2 years ago

3138 2266 2 years ago

openai.com

skilled2• 2 years ago

Live now,

OpenAI Spring Update (https://www.youtube.com/watch?v=DQacCB9tDaw)

https://news.ycombinator.com/item?id=40343950

EcommerceFlow3• 2 years ago

A new "flagship" model with no improvement of intelligence, very disappointed. Maybe this is a strategy for them to mass collect "live" data before they're left behind by Google/Twitter live data...

belter5• 2 years ago

https://youtu.be/DQacCB9tDaw

chzblck• 2 years ago

real time audio is mind blowing

throwup2384• 2 years ago

So what's the point of paying for ChatGPT Plus? And who on earth chose to make the app Mac only...

CSMastermind• 2 years ago

5x the capacity threshold is the only thing I heard them mention on the live stream.

Though presumably when they are ready to release new models the Plus users will get them first.

anuar12• 2 years ago

I think because usability increases so much (use cases of real-time conversation, and video-based coding, presentation feedback at work etc...) they would expect usage to drastically increase hence paying users would actually still have incentive to pay.

agd• 2 years ago

They mentioned an announcement about a new frontier model coming soon. Presumably this will be exclusive to paid users.

johnsimer• 2 years ago

Did they mention this in the gpt4o announcement video? I must have missed this

riffic2• 2 years ago

Plus users will have a message limit that is up to 5x greater than free users

from https://openai.com/index/gpt-4o-and-more-tools-to-chatgpt-fr...

tomschwiha• 2 years ago

I like the demo for sure more than the "reduced latency" Gemini demo [0].

[0] https://www.youtube.com/watch?v=UIZAiXYceBI

Powdering70822• 2 years ago

Wow this versioning scheme really messed up this prediction market: https://kalshi.com/markets/gpt4p5/gpt45-released

smusamashah• 2 years ago

That im-also-a-good-gpt2-chatbot[1] was in fact the new ChatGPT model as people were assuming few days ago here on HN[2].

Edit: may be not, name of that bot was just "gpt2-chatbot". May be that one was some initial iteration?

[1] https://twitter.com/LiamFedus/status/1790064963966370209/pho...

[2] https://news.ycombinator.com/item?id=40199715

theusus• 2 years ago

This 4o is already rolling out?

belter5• 2 years ago

They mentioned capabilities will be rolled out over the next few weeks: https://youtu.be/DQacCB9tDaw?t=5018

GalaxyNova2• 2 years ago

It is really cool that they are bringing this to free users. It does make me wonder what justifies ChatGPT plus now though...

InfiniteVortex• 2 years ago

they stated that they will be announcing something new that is on the next frontier (or close to it IIRC) soon. so there will definitely be an incentive to pay because it will be something better than gpt 4o.

pantsforbirds• 2 years ago

I assume the desktop app with voice and vision is rolling out to plus users first?

ppollaki• 2 years ago

I've noticed that the GPT-4 model's capabilities seem limited compared to its initial release. Others have also pointed this out. I suspect that making the model free might have required reducing its capabilities to meet cost efficiency goals. I'll have to try it out to see for myself.

EcommerceFlow3• 2 years ago

As I commented in the other thread, really really disappointed there's no intelligence update and more of a focus on "gimmicks". The desktop app did look really good, especially as the models get smarter. Will be canceling my premium as there's no real purpose of it until that new "flag ship" model comes out.

adroniser3• 2 years ago

Agree on hoping for an intelligence update, but I think it was clear from teasers that this was not gonna be GPT-5.

I'm not sure how fair it is to classify the new multimodal capabilities as just a gimmick though. I personally haven't integrated GPT-4 into my workflow that much and the latency and the fact I have to type a query out is a big reason why.

EcommerceFlow3• 2 years ago

After using it for an hour, I completely agree. Having the faster token rate + their excel analysis actually work is a game changer.

OutOfHere5• 2 years ago

I don't see 4o or anything new at https://platform.openai.com/docs/models

Overall I am highly skeptical of newer models as they risk worsening the completion quality to make them cheaper for OpenAI to run.

frabcus2• 2 years ago

It's there now! And still 128k context window

IanCal5• 2 years ago

It's there right now for me.

atgctg• 2 years ago

Tiktoken added support for GPT-4o: https://github.com/openai/tiktoken/commit/9d01e5670ff50eb74c...

It has an increased vocab size of 200k.

minimaxir3• 2 years ago

For posterity, GPT-3.5/4's tokenizer was 100k. The benefit of a larger tokenizer is more efficient tokenization (and therefore cheaper/faster) but with massive diminishing returns: the larger tokenizer makes the model more difficult to train but tends to reduce token usage by 10-15%.

simonw11• 2 years ago

Oh interesting, does that mean languages other than English won't be paying such a large penalty in terms of token lengths?

With previous tokenizers there was a notable increase in the number of tokens needed to represent non-English sentences: https://simonwillison.net/2023/Jun/8/gpt-tokenizers/

tedsanders2• 2 years ago

Yep. Non-English text gets a much bigger cost drop and speedup compared to English. Has always been a bummer that GPT-4 is like 5x slower and more expensive in Japanese, etc.

mike_hearn• 2 years ago

Does that imply they retrained the foundation model from scratch? I thought changing the tokenization was something you couldn't really retrofit to an existing model. I mean sure they might have initialized the weights from the prior GPT-4 model but it'd still require a lot of retraining.

famouswaffles24• 2 years ago

Yeah and they say as much in the blog.

moffkalast6• 2 years ago

Lots of those tokens would have to be pixel patches and sound samples right?

nojvek4• 2 years ago

Yep. Since it’s multimodal. Pictures, text, audio all go into token space.

kristofferR• 2 years ago

How are they able to use such a brand name, Tiktoken? Is it because TikTok is Chinese? Tiktoken, it's almost like if Apple released the Facebooken library for something entirely unrelated to Facebook.

gemeral• 2 years ago

That's not the right analogy. The "tok" in "Tiktoken" comes from "token", not "TikTok".

FergusArgyll• 2 years ago

First Impressions in no particular order:

  Being able to interrupt while GPT is talking
  2x faster/cheaper
  not really a much smarter model
  Desktop app that can see screenshots
  Can display emotions with and change the sound of "it's" voice

riffic2• 2 years ago

wondering what apple is cooking up and what they'll announce next month.

by the way the contraction "it's" is used to say "it is" or "it has", it is never a possessive form.

karaterobot2• 2 years ago

Unless you're talking about that sewer clown's balloon!

throwup2384• 2 years ago

Mac only desktop app. Windows version "later this year". No Linux.

Welp there goes my Plus subscription.

bmoxb• 2 years ago

It seems like a very odd decision. It's not like OpenAI can't afford to develop versions of the application for each OS in parallel.

unstatusthequo• 2 years ago

Why? Just use the API or normal web access version like you have been since ChatGPT became available at all.

ralusek2• 2 years ago

Can't find info which of these new features are available via the API

tazu• 2 years ago

Developers can also now access GPT-4o in the API as a text and vision model. GPT-4o is 2x faster, half the price, and has 5x higher rate limits compared to GPT-4 Turbo. We plan to launch support for GPT-4o's new audio and video capabilities to a small group of trusted partners in the API in the coming weeks.

ralusek2• 2 years ago

[EDIT] The model has since been added to the docs

Not seeing it or any of those documented here:

https://platform.openai.com/docs/models/overview

Jensson19• 2 years ago

The most impressive part is that the voice uses the right feelings and tonal language during the presentation. I'm not sure how much of that was that they had tested this over and over, but it is really hard to get that right so if they didn't fake it in some way I'd say that is revolutionary.

gdb• 2 years ago

(I work at OpenAI.)

It's really how it works.

xanderlewis5• 2 years ago

I like the humility in your first statement.

jamestimmins• 2 years ago

With this capability, how close are y'all to it being able to listen to my pronunciation of a new language (e.g. Italian) and given specific feedback about how to pronounce it like a local?

Seems like these would be similar.

baq4• 2 years ago

(I work at OpenAI.)

Winner of the 'understatement of the week' award (and it's only Monday).

Also top contender in the 'technically correct' category.

mttpgn• 2 years ago

Licensing the emotion-intoned TTS as a standalone API is something I would look forward to seeing. Not sure how feasible that would be if, as a sibling comment suggested, it bypasses the text-rendering step altogether.

skottenborg• 2 years ago

"(I work at OpenAI.)"

Ah yes, also known as being co-founder :)

terhechte• 2 years ago

Random OpenAI question: While the GPT models have become ever cheaper, the price for the tts models have stayed in the $15/1Mio char range. I was hoping this would also become cheaper at some point. There're so many apps (e.g. language learning) that quickly become too expensive given these prices. With the GPT-4o voice (which sounds much better than the current TTS or TTS HD endpoint) I thought maybe the prices for TTS would go down. Sadly that hasn't happened. Is that something on the OpenAI agenda?

9999000009992• 2 years ago

How far are we away from something like a helmet with chat GPT and a video camera installed, I imagine this will be awesome for low vision people. Imagine having a guide tell you how to walk to the grocery store, and help you grocery shop without an assistant. Of course you have tons of liability issues here, but this is very impressive

bjtitus• 2 years ago

Is it possible to use this as a TTS model? I noticed on the announcement post that this is a single model as opposed to a text model being piped to a separate TTS model.

cchance12• 2 years ago

This is damn near one of the most impressive things, can only imagine especially with live translation and voice synthesis (eleven labs style) you'd be capable of to integrate with something like teams (select each persons language and do realtime translation to each persons native language, with their own voice and intonations would NUTS)

rane• 2 years ago

Will the new voice mode allow mixing languages in sentences?

As a language learner, this would be tremendously useful.

selfmodruntime9• 2 years ago

I've always been wondering what GPT models lack that makes them "query->response" only. I've always tried to get chatbots to lose the initially needed query, with no avail. What would It take to get a GPT model to freely generate tokens in a thought like pattern? I think when I'm alone without query from another human. Why can't they?

ALittleLight• 2 years ago

In my ChatGPT app or on the website I can select GPT-4o as a model, but my model doesn't seem to work like the demo. The voice mode is the same as before and the images come from DALLE and ChatGPT doesn't seem to understand or modify them any better than previously.

jacobsimon2• 2 years ago

I couldn’t quite tell from the announcement, but is there still a separate TTS step, where GPT is generating tones/pitches that are to be used, or is it completely end to end where GPT is generating the output sounds directly?

rrr_oh_man4• 2 years ago

https://community.openai.com/t/when-i-log-in-to-chatgpt-i-am...

Sorry to hijack, but how the hell can I solve this? I have the EXACT SAME error on two iOS devices (native app only — web is fine), but not on Android, Mac, or Windows.

andybak• 2 years ago

May I just say this launch was a bit of a mess?

The web page implies you can try it immediately. Initially it wasn't available.

A few hours later it was in both the web UI and the mobile app - I got a popu[ telling me that GPT-4o was available. However nothing seems to be any different. I'm not given any option to use video as an input, the app can't seem to pick up any new info from my voice.

I'm left a bit confused as to what I can do that I couldn't do before. I certainly can't seem to recreate much of the stuff from the announcement demos.

dpflan• 2 years ago

Who's idea was the singing AIs? What specifically did you want to highlight with that part of the demo?

I imagine that there is a lot of usage at the HQ, human + AI karaoke?

hpeter2• 2 years ago

I can't wait to try it out, it sounds too good to be real.

It will be fully available in Eu with the GDPR compliance?

famouswaffles24• 2 years ago

The most impressive part is that the voice uses the right feelings and tonal language during the presentation.

Consequences of audio2audio (rather than audio >text text>audio). Being able to manipulate speech nearly as well as it manipulates text is something else. This will be a revelation for language learning amongst other things. And you can interrupt it freely now!

pants22• 2 years ago

However, this looks like it only works with speech - i.e. you can't ask it, "What's the tune I'm humming?" or "Why is my car making this noise?"

I could be wrong but I haven't seen any non-speech demos.

jcims2• 2 years ago

Anyone who has used elevenlabs for voice generation has found this to be the case. Voice to voice seems like magic.

twobitshifter• 2 years ago

I asked it to make a bird noise, instead it told me what a bird sounds like with words. True audio to audio should be able to be any noise, a trombone, traffic, a crashing sea, anything. Maybe there is a better prompt there but it did not seem like it.

bredren3• 2 years ago

I mention this down thread, but a symptom of a tech product of sufficient advancement is the nature of its introduction matters less and less.

Based on the casual production of these videos, the product must be this good.

https://news.ycombinator.com/item?id=40346002

jasondigitized11• 2 years ago

I noticed this as well. They gave zero fs about the fit and finish of these videos because they know this is magic in a bottle.

simonw11• 2 years ago

That was very impressive, but it doesn't surprise me much given how good the voice mode is in the ChatGPT iPhone app is already.

The new voice mode sounds better, but the current voice mode did also have inflection that made it feel much more natural than most computer voices I've heard before.

Jensson19• 2 years ago

Can you tell the current voice model what feelings and tone it should communicate with? If not it isn't even comparable, being able to control how it reads things is absolutely revolutionary, that is what was missing from using these AI models as voice actors.

bredren3• 2 years ago

The voice mode was quite good but the latency and start / stop has been encumbering.

duckmysick• 2 years ago

Slight off-topic, but I noticed you've updated your llm CLI app to work with the 4o model (plus bunch of other APIs through plugins). Kudos for working extremely fast. I'm really grateful for your tool; I tried many others, but for some reason none clicked as much as your.

Link in case other readers are curious: https://llm.datasette.io

newzisforsukas• 2 years ago

Right to who? To me, the voice sounds like an over enthusiastic podcast interviewer. Whats wrong with wanting computers to sound like what people think computers should sound like?

Jensson19• 2 years ago

It understands tonal language, you can tell it how you want it to talk, I have never seen a model like that before. If you want it to talk like a computer you can tell it to, they did it during the presentation, that is so much better than the old attempts at solving this.

navigate8310• 2 years ago

voice sounds like an over enthusiastic podcast interviewer

I believe it can be toned down using system prompts, which they'll expose in future iterations

tr3ntg4• 2 years ago

Right... enthusiastic and generally confused. It's uncanny valley level expressions. Still better than drab, monotonous speech though.

kybernetikos• 2 years ago

Genuine People Personalities™, just like in Hitchikers. Perhaps one of the milder forms of 'We Created The Torment Nexus'.

angryasian2• 2 years ago

agree I don't get it. I just want the right information and explained well. I don't want to be social with a robot.

Keyframe3• 2 years ago

It's a computer from the valley.

twelvechairs• 2 years ago

It sounds VERY California. "Its going great!" "Nice choice" "Whats up with the..." all within 10 seconds.

(not that this is the most important thing about the announcement at all. Just an aside)

exodust• 2 years ago

"over enthusiastic podcast interviewer"

Yeh it's cringe. I had to stop listening.

Why did they make the woman sound like she's permanently on the brink of giggling? It's nauseating how overstated her pretentious banter is. Somewhere between condescending nanny and preschool teacher. Like how you might talk to a child who's at risk of crying so you dial up the positive reinforcement.

danbala4• 2 years ago

exactly. Hope we can customize the voice soon. I want to talk to ultron... or the one from mass effect

mvkel2• 2 years ago

I was in the audience at the event. The only parts where it seemed to get snagged was hearing the audience reaction as an interruption. Which honestly makes the demo even better. It showed that hey, this is live.

Magic.

px433• 2 years ago

I wonder when it will be able to understand that there is more than one human talking to it. It seems like even in today's demo if two people are talking, it can't tell them apart.

ta-run3• 2 years ago

Crazy that interruption also seems to work pretty smoothly

tnias232• 2 years ago

Really? I think interruption and timing in general still seems like a problem that has yet to be solved. It was the most janky aspect of the demos imo.

nabakin• 2 years ago

Seems about as good as Azure's Speech Service. I wonder if that's what they are using behind the scenes

Keyframe3• 2 years ago

Somehow it also sounds almost like Dot Matrix from Spaceballs.

hnburnsy2• 2 years ago

Joan Rivers!

burntalmonds• 2 years ago

Yeah, the female voice especially is really impressive in the demos. The voice always sounds natural. The male voice I heard wasn't as good. It wasn't terrible, but it had a somewhat robotic feel to it.

Intralexical6• 2 years ago

"Right" feelings and tonal language? "Right" for what? For whom?

We've already seen how much damage dishonest actors can do by manipulating our text communications with words they don't mean, plans they don't intend to follow through on, and feelings they don't experience. The social media disinfo age has been bad enough.

Are you sure you want a machine which is able to manipulate our emotions on an even more granular and targetted level?

LLMs are still machines, designed and deployed by humans to perform a task. What will we miss if we anthropomorphize the product itself?

infinitezest2• 2 years ago

This gives me a lot of anxiety but my only recourse is to stop paying attention to AI dev. Its not going to stop, downside be damned. The "We're working super hard to make these things safe" routine from tech companies, who have largely been content to make messes and then not be held accountable in any significant way, rings pretty hollow for me. I don't want to be a doomer but I'm not convinced that the upside is good enough to protect us from the downside.

ec1096852• 2 years ago

I’m not sure how revolutionary the style is. It can already mimic many styles of writing. It seems like mimicking a cheerful happy assistant, with associated filler words, etc. is very much in-line with what LLM’s are good at.

noneeeed• 2 years ago

That's the part that really struck me. I thought it was particularly impressive with the Sal Khan maths tutor demo and the one with BeMyEyes. The comment at the end about the dog was an interesting ad-lib.

The only slightly annoying thing at the moment is they seem hard to interrupt, which is an important mechanism in conversations. But that seems like a solvable problem. They kind of need to be able to interpret body language a bit to spot when the speaker is about to interrupt.

modeless7• 2 years ago

As far as I'm concerned this is the new best demo of all time. This is going to change the world in short order. I doubt they will be ready with enough GPUs for the demand the voice+vision mode is going to get, if it's really released to all free users.

Now imagine this in a $16k humanoid robot, also announced this morning: https://www.youtube.com/watch?v=GzX1qOIO1bE The future is going to be wild.

andy99• 2 years ago

Really? If this was Apple it might make sense, for OpenAI it feels like a demo that's not particularly aligned with their core competency (a least by reputation) of building the most performant AI models. Or put another way, it says to me they're done building models and are now wading into territory where there are strong incumbents.

All the recent OpenAI talk had me concerned that the tech has peaked for now and that expectations are going to be reset.

modeless7• 2 years ago

What strong incumbents are there in conversational voice models? Siri? Google Assistant? This is in a completely different league. I can see from the reaction here that people don't understand. But they will when they try it.

Did you see it translate Italian? Have you ever tried the Google Translate/Assistant features for real time translation? They didn't train it to be a translator. They didn't make a translation feature. They just asked it. It's instantly better than every translation feature Google ever released.

golol9• 2 years ago

What Openai has done time and time again is completely change the landscape when the competitors have caught up and everyone thinks their lead is gone. They made image generation a thing. When GPT-3 became outdated they released ChatGPT. Instead of trying to keep Dalle competitive they released Sora. Now they change the game again with live audio+video.

10xDev3• 2 years ago

The future is not going to be anymore wild than what you choose to do with the tech.

modeless7• 2 years ago

I disagree completely. Even people who never adopt this stuff personally will have their lives profoundly impacted. The only way to avoid it would be to live in a large colony where the technology is prohibited, like the Amish. But even the Amish feel the influence of technology to some degree.

ilaksh3• 2 years ago

This is so amazing.. are there any open source models that are in any way comparable? Fully multimodal audio-to-audio etc.?

skilled2• 2 years ago

Parts of the demo were quite choppy (latency?) so this definitely feels rushed in response to Google I/O.

Other than that, looks good. Desktop app is great, but I didn’t see no mention of being able to use your own API key so OS projects might still be needed.

The biggest thing is bringing GPT-4 to free users, that is an interesting move. Depending on what the limits are, I might cancel my subscription.

Jordan-1172• 2 years ago

Seems like it was picking up on the audience reaction and stopping to listen.

To me the more troubling thing was the apparent hallucination (saying it sees the equation before he wrote it, commenting on an outfit when the camera was down, describing a table instead of his expression), but that might have just been latency awkwardness. Overall, the fast response is extremely impressive, as is the new emotional dimension of the voice.

sebastiennight2• 2 years ago

Aha, I think I saw the trick for the live demo: every time they used the "video feed", they did prompt the model specifically by saying:

- "What are you seeing now"

- "I'm showing this to you now"

etc.

The one time where he didn't prime the model to take a snapshot this way, was the time where the model saw the "table" (an old snapshot, since the phone was on the table/pointed at the table), so that might be the reason.

ayhanfuat2• 2 years ago

Commenting on the outfit was very weird indeed. Greg Brockman's demo includes some outfit related questions (https://twitter.com/gdb/status/1790071008499544518). It does seem very impressive though, even if they polished it on some specific tasks. I am looking forward to showing my desktop and asking questions.

tailspin20192• 2 years ago

Regarding the limits, I recently found that I was hitting limits very quickly on GPT-4 on my ChatGPT Plus plan.

I’m pretty sure that wasn’t always the case - it feels like somewhere along the lines the allowed usage was reduced, unless I’m imagining it. It wouldn’t be such a big deal if there was more visibility of my current usage compared to my total “allowance”.

I ended up upgrading to ChatGPT Team which has a minimum of 2x users (I now use both accounts) but I resented having to do this - especially being forced to pay for two users just to meet their arbitrary minimum.

I feel like I should not be hitting limits on the ChatGPT Plus paid plan at all based on my usage patterns.

I haven’t hit any limits on the Team plan yet.

I hope they continue to improve the paid plans and become a bit more transparent about usage limits/caps. I really do not mind paying for this (incredible) tech, but the way it’s being sold currently is not quite right and feels like paid users get a bit of a raw deal in some cases.

I have API access but just haven’t found an open source client that I like using as much as the native ChatGPT apps yet.

emporas8• 2 years ago

I use GPT from API in emacs, it's wonderful. Gptel is the program.

Although API access through Groq to Llama 3 (8b and 70b) is so much faster, that i cannot stand how slow GPT is anymore. It is slooow, still very capable model, but marginally better than open source alternatives.

gverrilla4• 2 years ago

I went through the exact same situation this last week. Didn't send more than 30 (token-heavy) messages within a few hours and it blocked me for 1 hour if I'm not wrong - paying user.

Jensson19• 2 years ago

Parts of the demo were quite choppy (latency?) so this definitely feels rushed in response to Google I/O.

It just stops the audio feed when it detects sound instead of an AI detecting when it should speak, so that part is horrible yeah. A full AI conversation would detect the natural pauses where you give it room to speak or when you try to take the word from it by interrupting, there it was just some dumb script to just shut it off when it hears sound.

But it is still very impressive for all the other part, that voice is really good.

Edit: If anyone from OpenAI reads this, at least fade out the voice quickly instead of chopping it, hard chopping off audio doesn't sound good at all, so many experienced this presentation to be extremely buggy due to it.

dharma1• 2 years ago

what's the download link for the desktop app? can't find it

mpeg• 2 years ago

seems like it might not be available for everyone? – my chatgpt plus doesn't show anything new, and also can't find the dekstop app

russdill• 2 years ago

They need to fade the audio or add some vocal queue when it's being interrupted. It makes it sound like it's losing connection. What'll be really impressive is when it intentionally starts interrupting you.

aantix3• 2 years ago

Agree. While watching the demo video, I thought I was the one having connectivity issues.

syntaxing• 2 years ago

I admit I drink the koolaid and love LLMs and their applications. But damn, the way it’s responds in the demo gave me goosebumps in a bad way. Like an uncanny valley instincts kicks in.

_Parfait_• 2 years ago

You're watching the species be reduced to an LLM.

warkdarrior• 2 years ago

Were humans an interesting species to start with, if they can be reduced to an LLM?

dougb5• 2 years ago

Hey that LLM is trained on everything we've ever produced, so I wouldn't say we've been "reduced", more like copied. I'll save my self-loathing for when a very low-parameter model can do this.

TheSockStealer• 2 years ago

I also thought the screwups, although minor, were interesting. Like when it thought his face was a desk because it did not update the image it was "viewing". It is still not perfect, which made the whole thing more believable.

mike006324• 2 years ago

I was shocked at how quickly and naturally they were able to correct the situation.

baobabKoodaa2• 2 years ago

Like when it thought his face was a desk because it did not update the image it was "viewing".

That's a rather uncharitable way of describing the situation. It didn't say anything like "your face looks like a wooden plank, it's very brown". It clearly understood that the image it was seeing was not matching the verbal request.

bbconn• 2 years ago

Yeah it made me realize that I actually don't want a human-like conversational bot (I have actual humans for that). Just teach me javascript like a robot.

bamboozled13• 2 years ago

Maybe it’s the geek in me, but I don’t want a talking computer.

I have enough talking people to deal with already .

wslack2• 2 years ago

It should do that, because it's still not actually an intelligence. It's a tool that is figuring out what to say in response that sounds intelligent - and will often succeed!

moffkalast6• 2 years ago

That kind of is an inteligence though. Chinese room meets solipsism and all that.

It is interesting how insanely close their demo is to the OSes in the movie "Her", it's basically a complete real life reproduction.

yCombLinks• 2 years ago

Welcome to half the people at your companies job.

HeatrayEnjoyer2• 2 years ago

It's more intelligent than many humans and most/all lesser animals. If it's not intelligent than I don't know what is.

isurujn• 2 years ago

The chuckling made me uneasy for some reason lol. Calm down, you're not like us. Don't pretend!

moffkalast6• 2 years ago

Can't wait for Meta's version 2 years down the line that someone will eventually fine tune to Agent Smith's personality and voice.

"Evolution, human. Evolution. Like the dinosaur. Look out that window. You've had your time. The future is our world. The future is our time."

unsupp0rted• 2 years ago

Yes, the chuckling was uncanny, but for me even more uncanny was how the female model went up at the end to soften what she was saying? into a question? even though it wasn't a question?

Eerily human female-like.

drivers99• 2 years ago

So I'm not the only one. Like I felt fear in a physical way. (Panic/adrenaline?) I'm sure I'd get used it but it was an interesting reaction. (I saw someone react that way to a talking Tandy 1000 once so, who knows.)

hubraumhugo• 2 years ago

The movie Her has just become reality

speedgoose2• 2 years ago

It’s getting closer. A few years ago the old Replika AI was already quite good as a romantic partner, especially when you started your messages with a * character to force OpenAI GPT-3 answers. You could do sexting that OpenAI will never let you have nowadays with ChatGPT.

aftbit3• 2 years ago

Why does OpenAI think that sexting is a bad thing? Why is AI safety all about not saying things that are disturbing or offensive, rather than not saying things that are false or unaligned?

volleygman180• 2 years ago

I was surprised that the voice is a ripoff of the AI voice in that movie (Scarlett Johansson) too

toxic722• 2 years ago

I am suspicious that they licensed Scarlet's voice for that voice model (Sky IIRC)

reducesuffering• 2 years ago

People realize where we're headed right? Entire human lives in front of a screen. Your online entertainment, your online job, your online friends, your online "relationship". Wake up, 12 hours screentime, eat food, go to bed. Depression and drug overdoses currently at sky high levels. Shocker.

emporas8• 2 years ago

If i can program with just my voice, there is no reason to not be in nature 10 hours a day minimum. My grandparent even slept outside as long as it was daytime.

Daytime is always a time to be outside, surrounded by many plants and stuff. It is a shame we have to be productive in some way, and most of production happens inside walls.

tr3ntg4• 2 years ago

Headed? We're there. Have been there. This just adds non-human sentient agents to the drama.

diimdeep• 2 years ago

It's been a reality in Russia since 2017.

The voice of "Alice" was dubbing actress Tatiana Shitova, who voiced most of Scarlett Johansson's characters and the voice of OS1, who called herself "Samantha", in the Russian dubbing of Spike Jonze's "Her".

https://en.wikipedia.org/wiki/Alice_(virtual_assistant)

hmmmhmmmhmmm• 2 years ago

With the news that Apple and OpenAI are closing / just closed a deal for iOS 18, it's easy to speculate we might be hearing about that exciting new model at WWDC...

thefourthchime• 2 years ago

Yes, i'm pretty sure this is the new Siri. Absolutely amazing, it's pretty much "here" from the movie.

hackerlight7• 2 years ago

Will this be available on old iPhones or only new ones going forward

chatcode3• 2 years ago

Parsing emotions in vocal inflections (and reliably producing them in vocal output) seems quite under-hyped in this release.

That seems to represent an entirely new depth of understanding of human reality.

deadbabe3• 2 years ago

Any appearance of understanding is just an illusion. It’s an LLM, nothing more.

chatcode3• 2 years ago

Sure, but that seems like it'll be a distinction without a difference for many use cases.

Having a reliable emotional model of a person based on their voice (or voice + appearance) can be useful in a thousand ways.

Which seems to represent a new frontier.

chpatrick2• 2 years ago

Any appearance of understanding is just an illusion. It’s just a pile of meat, nothing more.

rvz• 2 years ago

Given that they are moving all these features to free users, it tells us that GPT-5 is around the corner and is significantly much better than their previous models.

margorczynski• 2 years ago

Or maybe it is a desperation move after Llama 3 got released and the free mode will have such tight constraints that it will be unusable for anything a bit more serious.

PoignardAzur2• 2 years ago

Holy crap, the level of corporate cringe of that "two AIs talk to each other" scene is mind-boggling.

It feels like a pretty strong illustration of the awkwardness of getting value from recent AI developments. Like, this is technically super impressive, but also I'm not sure it gives us anything we couldn't have one year ago with GPT-4 and ElevenLabs.

sourcecodeplz• 2 years ago

It is quite nice how they keep giving premium features for free, after a while. I know openai is not open and all but damn, they do give some cool freebies.

BoumTAC• 2 years ago

Did they provide the limit rate for free user ?

Because I have the plus membership which is expensive (25$/month).

But if the limit is high enough (or my usage low enough), there is no point for paying that much money for me.

christianqchung• 2 years ago

Does anyone know how they're doing the audio part where Mark breaths too hard? Does his breathing get turned into all-caps text (AA EE OO) and that GPT4-o interprets that as him breathing too hard, or is there something more going on?

GalaxyNova2• 2 years ago

It can natively interpret voice now.

Jordan-1172• 2 years ago

That's how it used to do it, but my understanding is that this new model processes audio directly. If it were a music generator, the original would have generated sheet music to send to a synthesizer (text to speech), while now it can create the raw waveform from scratch.

modeless7• 2 years ago

There is no text. The model understands ingests audio directly and also outputs audio directly.

dclowd99013• 2 years ago

Is it a stretch to think this thing could accurately "talk" with animals?

reisse2• 2 years ago

So they retrained the whole model on audio datasets and the tokens are now sounds, not words/part of words?

crindy• 2 years ago

Very impressed by the demo where it starts speaking French in error, then laughs with the user about the mistake. Such a natural recovery.

spacebanana73• 2 years ago

We recognize that GPT-4o’s audio modalities present a variety of novel risks
For example, at launch, audio outputs will be limited to a selection of preset voices and will abide by our existing safety policies.

I wonder if they’ll ever allow truly custom voices from audio samples.

dkasper• 2 years ago

I think the issue there is less of a technical one and more of an issue with deepfakes and copyright

spacebanana73• 2 years ago

It might be possible to prove that I control my voice, or that of a given audio sample. For example by saying specific words on demand.

But yeah I see how they’d be blamed if anything went wrong, which it almost certainly would in some cases.

tomComb• 2 years ago

The price of 4o is 50% of GPT4-Turbo (and no mention of price change to gp4-turbo itself).

Given the competitive pressures I was expecting a much bigger price drop than that.

For non-multimodal uses, I don't think their API is at all competitive any more.

mrklol• 2 years ago

Where you get something cheaper with similar experience?

lagt_t• 2 years ago

Universal real time translation is incredibly dope.

I hate video players without volume control.

pachico• 2 years ago

jeez, that model really speaks a lot! I hope there's a way to make it more straight to the point rather than radio-like.

causal• 2 years ago

Clicking the "Try it on ChatGPT" link just takes me to GPT-4 chat window. Tried again in an incognito tab (supposing my account is the issue) and it just takes me to 3.5 chat. Anyone able to use it?

1010084• 2 years ago

Same here and also I can't hear audio in any of the videos on this page. Weird.

TrueDuality• 2 years ago

Weird visiting the page crashed my graphics driver using Firefox.

msoad• 2 years ago

They are admitting[1] that the new model is the gpt2-chatbot that we have seen before[2]. As many highlighted there, the model is not an improvement like GPT3->GPT4. I tested a bunch of programming stuff and it was not that much better.

It's interesting that OpenAI is highlighting the Elo score instead of showing results for many many benchmarks that all models are stuck at 50-70% success.

[1] https://twitter.com/LiamFedus/status/1790064963966370209

[2] https://news.ycombinator.com/item?id=40199715

modeless7• 2 years ago

"not that much better" is extremely impressive, because it's a much smaller and much faster model. Don't worry, GPT-5 is coming and it will be better.

TIPSIO• 2 years ago

Obviously given enough time there will always be better models coming.

But I am not convinced it will be another GPT-4 moment. Seems like big focus on tacking together multi-modal clever tricks vs straight better intelligence AI.

Hope they prove me wrong!

mupuff12342• 2 years ago

And how can one be so sure of that?

Seems to me that performance is converging and we might not see a significant jump until we have another breakthrough.

talldayo2• 2 years ago

Chalmers: "GPT-5? A vastly-improved model that somehow reduces the compute overhead while providing better answers with the same hardware architecture? At this time of year? In this kind of market?"

Skinner: "Yes."

Chalmers: "May I see it?"

Skinner: "No."

moomoo114• 2 years ago

I really hope GPT5 is good. GPT4 sucks at programming.

littlestymaar2• 2 years ago

I don't think a bigger model would make sense for OpenAI: it's much more important for them that they keep driving inference coat down, because there's no viable business model if they don't.

Improving the instruction tuning, the RLHF step, increase the training size, work on multilingual capabilities, etc. make sense as a way to improve quality, but I think increasing model size doesn't. Being able to advertize a big breakthrough may make sense in terms of marketing, but I don't believe it's going to happen for two reasons:

- you don't release intermediate steps when you want to be able to advertise big gains, because it raises the baseline and reduce the effectiveness of your ”big gains” in terms of marketing.

- I don't think they would benefit in an arm race with Meta, trying to keeping a significant edge. Meta is likely to be able to catch-up eventually on performance, but they are not so much of a threat in terms of business. Focusing on keeping a performance edge instead of making their business viable would be a strategic blunder.

cube22222• 2 years ago

I think the live demo that happened on the livestream is best to get a feel for this model[0].

I don't really care whether it's stronger than gpt-4-turbo or not. The direct real-time video and audio capabilities are absolutely magical and stunning. The responses in voice mode are now instantaneous, you can interrupt the model, you can talk to it while showing it a video, and it understands (and uses) intonation and emotion.

Really, just watch the live demo. I linked directly to where it starts.

Importantly, this makes the interaction a lot more "human-like".

[0] https://youtu.be/DQacCB9tDaw?t=557

gabiruh• 2 years ago

It's weird that the "airplane mode" seems to be ON on the phone during the entire presentation.

fvdessen4• 2 years ago

The demo is impressive but personally, as a commercial user, for my practical use cases, the only thing I care about is how smart it is, how accurate are its answers and how vast is its knowledge. These haven’t changed much since GPT-4, yet they should, as IMHO it is still borderline in its abilities to be really that useful

aaroninsf• 2 years ago

Absolutely agree.

This model isn't about basemark chasing or being a better code generator; it's entirely explicitly focused on pushing prior results into the frame of multi-modal interaction.

It's still a WIP, most of the videos show awkwardness where its capacity to understand the "flow" of human speech is still vestigial. It doesn't understand how humans pause and give one another space for such pauses yet.

But it has some indeed magic ability to share a deictic frame of reference.

I have been waiting for this specific advance, because it is going to significantly quiet the "stochastic parrot" line of wilfully-myopic criticism.

It is very hard to make blustery claims about "glorified Markov token generation" when using language in a way that requires both a shared world model and an understanding of interlocutor intent, focus, etc.

This is edging closer to the moment when it becomes very hard to argue that system does not have some form of self-model and a world model within which self, other, and other objects and environments exist with inferred and explicit relationships.

This is just the beginning. It will be very interesting to see how strong its current abilities are in this domain; it's one thing to have object classification—another thing entirely to infer "scripts plans goals..." and things like intent, and, deixis. E.g. how well does it now understand "us" and "them" and "this" vs "that"?

Exciting times. Scary times. Yee hawwwww.

snthpy• 2 years ago

Hectic!

Thanks for this.

OJFord3• 2 years ago

I assume (because they don't address it or look at all phased) the audio cutting in and out is just an artefact of the stream?

mvdtnz• 2 years ago

Interesting that they decided to keep the horrible ChatGPT tone ("wow you're doing a live demo right now?!"). It comes across just so much worse in voice. I don't need my "AI" speaking to me like I'm a toddler.

ChuckMcM2• 2 years ago

I expect the really solid use case here will be voice interfaces to applications that don't suck. Something I am still surprised at is that vendors like Apple have yet to allow me to train the voice to text model so that it only responds to me and not someone else.

So local modelling (completely offline but per speaker aware and responsive), with a really flexible application API. Sort of the GTK or QT equivalent for voice interactions. Also custom naming, so instead of "Hey Siri" or "Hey Google" I could say, "Hey idiot" :-)

Definitely some interesting tech here.

spaceman_20203• 2 years ago

This is going straight into 'Her' territory

clhodapp3• 2 years ago

Call me overly paranoid/skeptical, but I'm not convinced that this isn't a human reading (and embellishing) a script. The "AI" responses in the script may well have actually been generated by their LLM, providing a defense against it being fully fake, but I'm just not buying some of these "AI" voices.

We'll have to see when end users actually get access to the voice features "in the coming weeks".

dragonwriter7• 2 years ago

As many highlighted there, the model is not an improvement like GPT3->GPT4.

The improvements they seem to be hyping are in multimodality and speed (also price – half that of GPT-4 Turbo – though that’s their choice and could be promotional, but I expect it’s at least in part, like speed, a consequence of greater efficiency), not so much producing better output for the same pure-text inputs.

kybercore• 2 years ago

the model scores 60 points higher in lmsys than the best gpt 4 turbo model from april, that's still a pretty significant jump in text capability

aixpert• 2 years ago

useless anecdata but I find the new model very frustrating, often completely ignoring what I say in follow up queries. it's giving me serious Siri vibes

(text input in web version)

maybe it's programmed to completely ignore swearing but how could I not swear after it gave me repeatedly info about you.com when I try to address it in second person

lossolo• 2 years ago

I agree. I tried a few programming problems that, let's say, seem to be out of the distribution of their training data and which GPT4 failed to solve before. The model couldn't find a similar pattern and failed to solve them again. What's interesting is that one of these problems were solved by Opus, which seems to indicate that the majority of progress in the last months should be attributed to the quality/source of the training data.

avereveard• 2 years ago

I tested a few use cases in the chat, and it's not particularly more intelligent but they seem to have solved laziness. I had to categorize my expenses to do some budgeting for the family, and in gpt 4 I had to go ten in ten, confirm the suggested category, download the file, took two days as I was constantly hitting the limit. gpt4o did most of the grunth work, then commincated anomalies in bulk, asked for suggestion for these, and provided a downloadable link in two answers, calling the code interpreter mulitple times, and working toward the goal on it's own.

and the prompt wasn't a monstrosity, and it wasn't even that good, it was just one line "I need help to categorize these expenses" and off it went. hope it won't get enshittified like turbo, because this finally feels as great as 3.5 was for goal seeking.

ozzydave• 2 years ago

Heh - I'm using ChatGPT for the same thing! Works 10X better than Rocket Money, which was supposed to be an improvement on Mint but meh.

jameshart12• 2 years ago

I think this comment is easily misread as implying that this GPT4o model is based on some old GPT2 chatbot - that’s very much not what you meant to say, though.

This model has been being tested under a code name of ‘gpt2-chatbot’ but it is very much a new GPT4+-level model, with new multimodal capabilities - but apparently some impressive work around inference speed.

Highlighting so people don’t get the impression this is just OpenAI slapping a new label on something a generation out of date.

vitorgrs3• 2 years ago

They are admitting that is the im-also-a-good-gpt2-chatbot. There was 3.... Don't ask me why.

The "gpt2-chatbot" was the worst of the three.

Jimmc414• 2 years ago

Big questions are (1) when is this going to be rolled out to paid users? (2) what is the remaining benefit of being a paid user if this is rolled out to free users? (3) Biggest concern is will this degrade the paid experience since GPT-4 interactions are already rate limited. Does OpenAI have the hardware to handle this?

Edit: according to @gdb this is coming in "weeks"

https://twitter.com/gdb/status/1790074041614717210

onemiketwelve• 2 years ago

thanks, I was confused because the top of the page says to try now when you cannot in fact try it at all

caturopath• 2 years ago

Yeah, it's weird. Confused me too.

freedomben• 2 years ago

I'm a ChatGPT Plus Subscriber and I just refreshed the page and it offered me the new model. I'm guessing they're rolling it out gradually but hopefull it won't take too long.

Edit: It's also now available to me in the Android App

whimsicalism14• 2 years ago

i can try it now, but now the voice features i dont think

zamadatix3• 2 years ago

You can use GPT-4o now but the interactive voice mode of using it (as demoed today) releases in a few weeks.

Tenoke• 2 years ago

what is the remaining benefit of being a paid user if this is rolled out to free users?

It says so right in the post

We are making GPT-4o available in the free tier, and to Plus users with up to 5x higher message limits

The limits are much lower for free users.

jrh3• 2 years ago

I'm not convinced I need to keep paying for plus. The threshold of requests for free 4o is pretty high.

dunkmaster2• 2 years ago

This might mean GPT-5 is coming soon and it will only be available to paid users.

dunkmaster2• 2 years ago

Or they just made a bunch of money on their licensing deal with Apple. So they don't need to charge for ChatGPT anymore.

yieldcrv3• 2 years ago

I'm actually thinking that the GPT store with more users might be better for them

From my casual conversations, not that many people are paying for GPT4 or know why they should. Every conversation even in enthusiast forums like this one has to be interjected with "wait, are you using GPT4? because GPT3.5 the free one is pretty nerfed"

just nuking that friction from orbit and expanding the GPT store volume could be a positive for them

lxgr3• 2 years ago

Will this include image generation for the free tier as well? That's a big missing feature in OpenAI's free tier compared to Google and Meta.

dkarras2• 2 years ago

is oai image generation any different than the microsoft copilot provides for free? I thought they were the same.

lxgr3• 2 years ago

Oh, I meant the actual ChatGPT service, not just something powered by GPT-4 or 3.5.

I've found Microsoft Copilot to be somewhat irritating to work with – I can't really put my finger on it, but it seems to be resorting to Bing search and/or the use of emoji in its replies a bit too much.

OliverM• 2 years ago

This is impressive, but they just sound so _alien_, especially to this non-U.S. English speaker (to the point of being actively irritating to listen to). I guess picking up on social cues communicating this (rather than express instruction or feedback) is still some time away.

It's still astonishing to consider what this demonstrates!

w-m• 2 years ago

Gone are the days of copy-pasting to/from ChatGPT all the time, now you just share your screen. That's a fantastic feature, in how much friction that removes. But what an absolute privacy nightmare.

With ChatGPT having a very simple text+attachment in, text out interface, I felt absolutely in control of what I tell it. Now when it's grabbing my screen or a live camera feed, that will be gone. And I'll still use it, because it's just so damn convenient?

baby_souffle• 2 years ago

Now when it's grabbing my screen or a live camera feed, that will be gone. And I'll still use it, because it's just so damn convenient?

Presumably you'll have a way to draw a bounding box around what you want to show or limit to just a particular window the same way you can when doing a screen share w/ modern video conferencing?

jawiggins• 2 years ago

I hope when this gets to my iphone I can use it to set two concurrent timers.

mellosouls2• 2 years ago

Very, very impressive for a "minor" release demo. The capabilities here would look shockingly advanced just 5 years ago.

Universal translator, pair programmer, completely human sounding voice assistant and all in real time. Scifi tropes made real.

But: Interesting next to see how it actually performs IRL latency and without cherry-picking. No snark, it was great but need to see real world power. Also what the benefits are to subscribers if all this is going to be free...

llm_trw8• 2 years ago

The capabilities here look shocking advanced yesterday.

partiallypro7• 2 years ago

A lot of the demo is very impressive, but some of it is just stuff that already exists but this is slightly more polished. Not really a huge leap for at least 60% of the demos.

CooCooCaCha9• 2 years ago

My guess is they're banking on the free version being rate limited and people finding it so useful that they want to remove the limit. Like giving a new user a discount on heroin. At least that's the strategy that would make most sense to me.

rubidium• 2 years ago

I have the paid version and it’s not connecting

ipnon• 2 years ago

The free tier is just to get more training data.

AuryGlenz• 2 years ago

I don’t know why they didn’t do that a long time ago (apart from limited hardware). So many people have probably tried GPT3.5 and bounced off unimpressed.

LoveMortuus• 2 years ago

I'm guessing it's going to be free only for a limited time. Probably until they have enough voice data to improve the model.

I can't really think of any other reason why they would make this free, especially when they already have a premium plan.

yumraj• 2 years ago

In the first video the AI seems excessively chatty.

hipadev23• 2 years ago

chatGPT desperately needs a "get to the fucking point" mode.

tomashubelbauer• 2 years ago

Seriously. I've had to spell out that it should just answer in twelve different ways with examples in the custom instructions to make it at least somewhat usable. And it still "forgets" sometimes.

chatcode3• 2 years ago

It does, that's "custom instructions".

progbits• 2 years ago

Impressive demo, but like half the interactions were "hello" "hi how are you doing" "great thanks, what can I help you with" etc.

The benchmark for human-computer interaction should be "tea, earl gray, hot", not awkward and pointless smalltalk.

ativzzz• 2 years ago

"no yapping" in the prompt works very well

jamilton2• 2 years ago

Yeah, I would hope that custom instructions would help somewhat with that, but it is a point of annoyance for me too.

mrandish• 2 years ago

Yes, it sounds like an awkwardly perky and over-chatty telemarketer that really wants to be your friend. I find the tone maximally annoying and think most users will find it both stupid and creepy. Based on user preferences, I expect future interactive chat AIs will default to an engagement mode that's optimized for accuracy and is both time-efficient and cognitively efficient for the user.

I suspect this AI <-> Human engagement style will evolve over time to become quite unlike human to human engagement, probably mixing speech with short tones for standard responses like "understood", "will do", "standing by" or "need more input". In the future these old-time demo videos where an AI is forced to do a creepy caricature of an awkward, inauthentic human will be embarrassingly retro-cringe. "Okay, let's do it!"

jdthedisciple• 2 years ago

I found i off-putting as well

guess it's just biased with average Californian behavior and speech patterns

TillE• 2 years ago

Reminds me of how Siri used to make jokes after setting a timer. Now it just reads back the time you specified, in a consistent way.

It's a very impressive gimmick, but I really think most people don't want to interact with computers that way. Since Apple pulled that "feature" after a few years, it's probably not just a nerd thing.

caseyy4• 2 years ago

It is exceptionally creepy. It is an unnatural effort to appear pleasing, like the fawning response seen in serious abuse survivors.

csjh• 2 years ago

I wonder if this is what the "gpt2-chatbot" that was going around earlier this month was

lambdaba• 2 years ago

yes it was

AndyNemmity• 2 years ago

it was

peppertree• 2 years ago

Just like that Google is on back foot again.

tempsy• 2 years ago

Considering the stock pumped following the presentation the market doesn't seem particularly with what OpenAI released at all.

htrp3• 2 years ago

The market was expecting ChatGPT search to take a chunk out of Google's search revenue.

sebastiennight2• 2 years ago

Anyone who watched the OpenAI livestream: did they "paste" the code after hitting CTRL+C ? Or did the desktop app just read from the clipboard?

Edit: I'm asking because of the obvious data security implications of having your desktop app read from the clipboard _in the live demo_... That would definitely put a damper to my fanboyish enthusiasm about that desktop app.

golol9• 2 years ago

To me it looked they used one command that did copy+paste into ChatGPT both.

dkarras2• 2 years ago

macOS asks you to give permission for an application to read your clipboard. do other operating systems not have that?

sn_master• 2 years ago

This is every romance scammer's dreams come true...

summerlight2• 2 years ago

This is really impressive engineering. I thought real time agents would completely change the way we're going to interact with large models but it would take 1~2 more years. I wonder what kind of new techs are developed to enable this, but OpenAI is fairly secretive so we won't be able to know their sauce.

On the other hand, this also feels like a signal that reasoning capability has probably already been plateaued at GPT-4 level and OpenAI knew it so they decided to focus on research that matters to delivering product engineering rather than long-term research to unlock further general (super)intelligence.

nopinsight2• 2 years ago

Reliable agents in diverse domains need better reasoning ability and fewer hallucinations. If the rumored GPT-5 and Q* capabilities are true, such agents could become available soon after it’s launched.

summerlight2• 2 years ago

Sam has been pretty clear on denying GPT-5 rumors, so I don't think it will come anytime soon.

MVissers3• 2 years ago

Why would reasoning have plateau’d?

I think reasoning ability is not the largest bottleneck for improvement in usefulness right now. Cost is a bigger one IMO.

Running these models as agents is hella expensive, and agents or agent-like recurrent reasoning (like humans do) is the key to improved performance if you look at any type of human intelligence.

Single-shot performance only gets you so far.

For example- If it can write code 90% of the way, and then debug in a loop, it’d be much more performant than any single shot algorithm.

And OpenAI has these huge models in their basement probably. But they might not be much more useful than GPT-4 when used as single-shot. I mean, what could it do what we can’t do today with gpt-4?

It’s agents and recurrent reasoning we need for more usefulness.

At least- That’s my humble opinion as an amateur neuroscientist that plays around with these models.

Jensson19• 2 years ago

Running these models as agents is hella expensive

Because they are dumb so you need to over compute so many things to get anything useful. Smarter models would solve this problem. Making the current model cheaper is like trying to solve Go by scaling up Deep Blue, it doesn't work to just hardcode dumb pieces together, the model needs to get smarter.

Aeolun3• 2 years ago

I’d be pretty happy if they could just make ChatGPT4 10x faster and cheaper. It’d be fine for basically all of my use cases.

meet_zaveri• 2 years ago

For example- If it can write code 90% of the way, and then debug in a loop, it’d be much more performant than any single shot algorithm.

OOC, Would this make the academics including algorithms as more or less important in their curriculum? That's a bad win for soceity if it's true.

CuriouslyC• 2 years ago

This isn't really new tech, it's just an async agent in front of a multimodal model. It seems from the demo that the improvements have been in response latency and audio generation. Still, it looks like they're building a solid product, which has been their big issue so far.

searealist• 2 years ago

No, audio is fed directly into the model. There is no text to speech transformer in front of it like there was with chatgpt-4.

cchance12• 2 years ago

Its 200-300ms for a multimodal response, thats REALLY a big step forward, especially given it's doing it with full voice response, not just text.

cchance12• 2 years ago

Ya so sad that OpenAI isn't more Open imagine if OpenAI was still sharing their thought processes and papers with the overall commity, really wish we saw collaborations between OpenAI and Meta for instance to really have helped push the open source arena further ahead, i love that their latest models are so great but the fact they aren't helping the Open source arena to progress is sad. Imagine how far we'd be if OpenAI was still as open as they once were and we saw collaborations betweeen Meta, OpenAI and Anthropic all working and sharing growth and tech to reduce double work and help each other not go down failed paths.

MBCook2• 2 years ago

Why must every website put stupid stuff that floats above the content and can’t be dismissed? It drives me nuts.

dkga• 2 years ago

That can “reason”?

MisterBiggs2• 2 years ago

I've been waiting to see someone drop a desktop app like they showcased. I wonder how long until it is normal to have an AI looking at your screen the entire time your machine is unlocked. Answering contextual questions and maybe even interjecting if it notices you made a mistake and moved on.

doomroot13• 2 years ago

That seems to be what Microsoft is building and will reveal as a new Windows feature at BUILD '24. Not too sure about the interjecting aspect but ingesting everything you do on your machine so you can easily recall and search and ask questions, etc. AI Explorer is the rumored name and will possibly run locally on Qualcomm NPUs.

ukuina3• 2 years ago

Yes, this is Windows AI Explorer.

layer87• 2 years ago

This will be great for employee surveillance, to monitor how much you are really working.

MisterBiggs2• 2 years ago

I think even scarier is that ChatGPT’s tone of voice and bias is going to take over everything.

bredren3• 2 years ago

It is notable OpenAI did not need to carefully rehearse the talking points of the speakers. Or even do the kind of careful production quality seen in a lot of other videos.

The technology product is so good and so advanced it doesn't matter how the people appear.

Zuck tried this in his video countering to vision pro, but it did not have the authentic "not really rehearsed or produced" feel of this at all. If you watch that video and compare it with this you can see the difference.

Very interesting times.

skepticATX3• 2 years ago

Very impressive demo, but not really a step change in my opinion. The hype from OpenAI employees was on another level, way more than was warranted in my opinion.

Ultimately, the promise of LLM proponents is that these models will get exponentially smarter - this hasn’t born out yet. So from that perspective, this was a disappointing release.

If anything, this feels like a rushed release to match what Google will be demoing tomorrow.

altcognito2• 2 years ago

GPT-4 expressing a human-like emotional response every single time you interact with it is pretty annoying.

In general, trying to push that this is a human being is probably "unsafe", but that hurts the marketing.

jonquark• 2 years ago

It might be region specific (I'm in the UK) - but I don't "see" the new model anywhere e.g. if I go to: https://platform.openai.com/playground/chat?models=gpt-4o The model the page uses is set to gpt-3.5-turbo-16k.

I'm confused

aw4y• 2 years ago

I don't see anything released today. Login/signup is still required, no signs of desktop app or free use on web. What am I missing?

goalonetwo• 2 years ago

For all the hype around this announcement I was expecting more than some demo-level stuff that close to nobody will use in real life. Disappointing.

sroussey3• 2 years ago

Twice as fast and half the cost for the API sounds good to me. Not a demoable thing though.

asteroidz• 2 years ago

Why are you so confident that nobody will use this in real life? I know OpenAI showed only a few demos, but I can see huge potential.

sensanaty2• 2 years ago

That's not true, scammers will definitely be using this a lot! Also clueless C-levels who want to nix hundreds of human customer support agents!

You'll get to sit on the phone talking to some convincing robot that won't let you do anything so that the megacorps can save 0.0001 cents! Ain't progress looking so good?

mellosouls2• 2 years ago

@sama reflects:

https://blog.samaltman.com/gpt-4o

1010084• 2 years ago

Are the employees in the demo high-directives of OpenAI? I can understand Altman being happy with this progress, but what about the medium/low employees? Didn't they watch Oppenheimer? Are they happy they are destroying humanity/work/etc for future and not-so-future generations?

Anyone who thinks this will be like the previous work revolutions is nonsense. This replaces humans and will replace them even more on each new advance. What's their plan? Live out of their savings? What about family/friends? I honestly can't see this and think how they can be so happy about it...

"Hey, we created something very powerful that will do your work for free! And it does it better than you and faster than you! Who are you? It doesn't matter, it applies to all of you!"

And considering I was thinking in having a kid next year, well, this is a no.

galdosdi5• 2 years ago

Have a kid anyway, if you otherwise really felt driven to it. Reading the tealeaves in the news is a dumb reason to change decisions like that. There's always some disaster looming, always has been. If you raise them well they'll adapt well to whatever weird future they inherit and be amongst the ones who help others get through it

1010084• 2 years ago

Thanks for taking the time to answer instead of (just) downvoting. I understand your logic but I don't see a future where people can adapt to this and get through it. I honestly see a future so dark and we'll be there much sooner than we thought... when OpenAI released their first model people were talking about years before seeing real changes and look what happened. The advance is exponential...

nice_byte• 2 years ago

"It is difficult to get a man to understand something when his salary depends on his not understanding it."

karaterobot2• 2 years ago

That first demo video was impressive, but then it ended very abruptly. It made me wonder if the next response was not as good as the prior ones.

dclowd99013• 2 years ago

Extremely impressive -- hopefully there will be an option to color all responses with a underlying brevity. It seemed like the AI just kept droning on and on.

MP_172913• 2 years ago

This thing continues to stress my skepticism for AI scaling laws and the broad AI semiconductor capex spending.

1- OpenAI is still working in GPT-4-level models. More than 14 months after the launch of GPT-4 and after more than $10B in capital raised. 2- The rhythm that token prices are collapsing is bizarre. Now a (bit) better model for 50% of the price. How people seriously expect these foundational model companies to make substantial revenue? Token volume needs to double just for revenue to stand still. Since GPT-4 launch, token prices are falling 84% per year!! Good for mankind, but crazy for these companies. 3- Maybe I am an asshole, but where are my agents? I mean, good for the consumer use case. Let's hope the rumors that Apple is deploying ChatGPT with Siri are true, these features will help a lot. But I wanted agents! 4- These drop in costs are good for the environment! No reason to expect them to stop here.

htrp3• 2 years ago

Did we ever get confirmation that GPT 4 was a fresh training run vs increasingly complex training on more tokens on the base GPT3 models?

saliagato• 2 years ago

gpt-4 was indeed trained on gpt-3 instruct series (davinci, specifically). gpt-4 was never a newly trained model

ldjkfkdsjnv• 2 years ago

Yeah I'm also getting suspicious. Also, all of the models (opus, llama3, gpt4, gemini pro) are converging to similar levels of performance. If it was true that the scaling hypothesis was true, we would see a greater divergence of model performance

bigyikes12• 2 years ago

Plot model performance over the last 10 years and show me where the convergence is.

The graph looks like an exponential and is still increasing.

Every exponential is a sigmoid in disguise, but I don’t think there has been enough time to say the curve has flattened.

hehdhdjehehegwv4• 2 years ago

This is why think Meta has been so shrewd in their “open” model approach. I can run Llama3-70B on my local workstation with an A6000, which after the up-front cost of the card, is just my electricity bill.

So despite all the effort and cost that goes into these models, you still have to compete against a “free” offering.

Meta doesn’t sell an API, but they can make it harder for everybody else to make money on it.

kmeisthax2• 2 years ago

LLaMA still has an "IP hook" - the license for LLaMA forbids usage on applications with large numbers of daily active users, so presumably at that point Facebook can start asking for money to use the model.

Whether or not that's actually enforceable[0], and whether or not other companies will actually challenge Facebook legal over it, is a different question.

[0] AI might not be copyrightable. Under US law, copyright only accrues in creative works. The weights of an AI model are a compressed representation of training data. Compressing something isn't a creative process so it creates no additional copyright; so the only way one can gain ownership of the model weights is to own the training data that gets put into them. And most if not all AI companies are not making their own training data...

spacebanana73• 2 years ago

Sam Altman gave the impression that foundation models would be a commodity on his appearance in the All in Podcast, at least in my read of what he said.

The revenue will likely come from application layer and platform services. ChatGPT is still much better tuned for conversation than anything else in my subjective experience and I’m paying premium because of that.

Alternatively it could be like search - where between having a slightly better model and getting Apple to make you the default, there’s an ad market to be tapped.

hn_throwaway_994• 2 years ago

I'm ceaselessly amazed at people's capacity for impatience. I mean, when GPT 4 came out, I was like "holy f, this is magic!!" How quickly we get used to that magic and demand more.

Especially since this demo is extremely impressive given the voice capabilities, yet still the reaction is, essentially, "But what about AGI??!!" Seriously, take a breather. Never before in my entire career have I seen technology advance at such a breakneck speed - don't forget transformers were only invented 7 years ago. So yes, there will be some ups and downs, but I couldn't help but laugh at the thought that "14 months" is seen as a long time...

belter5• 2 years ago

Chair in the sky again...

bamboozled13• 2 years ago

You just be new here?

tsunamifury2• 2 years ago

It's pretty bizarre how these demos bring out keyboard warriors and cereal bowl yellers like crazy. Huge breakthroughs in natural cadence, tone and interaction as well as realtime mutlimodal and all the people on HN can rant about is token price collapse

It's like the people in this community all suffer from a complete disconnect from society and normal human needs/wants/demands.

ThrowawayTestr• 2 years ago

How quickly we get used to that magic and demand more.

Humanity in a nutshell.

seydor2• 2 years ago

We re just logarithmic creatures

MP_172913• 2 years ago

I am just talking about scaling laws and the level of capex that big tech companies are doing. One hundred billion dollars are being invested this year to pursue AI scaling laws.

You can be excited, as I am, while also being bearish, as I am.

ertgbnm2• 2 years ago

Over a year they have provided an order of magnitude improvements on latency, context length, and cost, while meaningfully improving performance and adding several input and output modalities.

financypants• 2 years ago

There are well talked about cons to shipping so fast, but on the bright side, when everyone is demanding more, more, more, it pushes cost down and demands innovation, right?

015a2• 2 years ago

Peoples' "capacity for impatience" is literally the reason why these things move so quick. These are not feelings at-odds with each other; they're the same thing. Its magical; now its boring; where's the magic; let's create more magic.

Be impatient. Its a positive feeling, not a negative one. Be disappointed with the current progress; its the biggest thing keeping progress moving forward. It also, if nothing else, helps communicate to OpenAI whether they're moving in the right direction.

fnordpiglet9• 2 years ago

IMO, for fear of being label a hype boy, this is absolutely a sign of the impending singularity. We are taking an ever accelerating frame of cultural reference as a given and our expectation is that exponential improvement is not just here but you’re already behind once you’ve released.

I spend the last two years dismayed with the reaction but I’ve just recently begun to realize this is a feature not a flaw. This is latent demand for the next iteration expressed as impatient dissatisfaction with the current rate of change inducing a faster rate of change. Welcome to the future you were promised.

laweijfmvo• 2 years ago

Sounds like the Jeopardy answer for "What is a novelty?"

spaceman_20203• 2 years ago

People fume and fret about startups wasting capital like it was their own money.

GPT and all the other chatbots are still absolutely magic. The idea that I can get a computer to create a fully functional app is insane.

Will this app make me millions and run a business? Probably not. Does it do what I want it to do? Mostly yes.

nsagent• 2 years ago

Reminds me of the Louis CK bit about internet:

https://youtube.com/watch?v=me4BZBsHwZs

IanCal5• 2 years ago

Tbf gpt4 level seems useful and better than almost everything else (or close if not). The more important barriers for use in applications have been cost, throughout and latency. Oh and modalities, which have expanded hugely.

adtac• 2 years ago

Token volume needs to double just for revenue to stand still

Profits are the real metric. Token volume doesn't need to double for profits to stand still if operational costs go down.

mrkramer2• 2 years ago

This thing continues to stress my skepticism for AI scaling laws and the broad AI semiconductor capex spending.

Imagine you are in 1970s and saying computers suck, they are expensive, there is not that many use cases....fast forward to 90s and you are using Windows 95 with GUI and chip astronomically more powerful that we had in 70s and you can use productivity apps , play video games and surf Internet.

Give AI time, it will fulfill its true protentional sooner or later.

MP_172913• 2 years ago

That's the opposite of what I am saying.

What I am saying is that computers are SO GOOD that AI is getting VERY CHEAP and the amount of computing capex being done is excessive.

It's more like you are in 1999, people are spending $100B in fiber, while a lot of computer scientists are working in compression, multiplexing, etc.

madeofpalk• 2 years ago

Token volume needs to double just for revenue to stand still

I'm pretty skeptical about all the whole LLM/AI hype, but I also believe that the market is still relatively untapped. I'm sure Apple switching Siri to an LLM would ~double token usage.

A few products rushed out thin wrappers ontop of chatgpt ai, developing pretty uninspiring chat bots of limited use. I think there's still huge potential for this LLM technology to be 'just' an implementation detail of other features, just running in the background doing its thing.

That said, I don't think OpenAI has much of a moat here. They were first, but there's plenty of others with closed or open models.

drag0s• 2 years ago

what do you actually expect from an "agent"?

MP_172913• 2 years ago

Ask stuff like "Check whether there's some correlation between the major economies fiscal primary deficit and GDP growth in the post-pandemic era" and get an answer.

Pr0ject217• 2 years ago

"OpenAI is still working in GPT-4-level models."

This may or may not be true - just because we haven't seen GPT-level-5 capabilities, does not mean that it does not yet exist. It is highly unlikely that what they ship is actually the full capability of what they have access to.

MP_172913• 2 years ago

they literally launched TODAY a GPT-4 model!

bionhoward• 2 years ago

imho gpt4 is definitely [proto-]agi and the reason i cancelled my openai sub and am sad to miss out on talking to gpt4o is, openai thinks it's illegal, harmful, or abusive to use their model output to develop models that compete with openai. which means if you use openai then whatever comes out of it is toxic waste due to an arguably illegal smidgen of legal bullshit.

for another adjacent example, every piece of code github copilot ever wrote, for example, is microsoft ai output, which you "can't use to develop / otherwise improve ai," some nonsense like that.

the sum total of these various prohibitions is a data provenance nightmare of extreme proportion we cannot afford to ignore because you could say something to an AI and they parrot it right back to you and suddenly the megacorporation can say that's AI output you can't use in competition with them, and they do everything, so what can you do?

answer: cancel your openai sub and shred everything you ever got from them, even if it was awesome or revolutionary, that's the truth here, you don't want their stuff and you don't want them to have your stuff. think about the multi-decade economics of it all and realize "customer noncompete" is never gonna be OK in the long run (highway to corpo hell imho)

fnordpiglet9• 2 years ago

Where I work in the hoary fringes of high end tech we can’t secure enough token processing for our use cases. Token price decreases means opening of capacity but we immediately hit the boundaries of what we can acquire. We can’t keep up with the use cases - but more than that we can’t develop tooling to harness things fast enough and the tooling we are creating is a quick hack. I don’t fear for the revenue of base model providers. But I think in the end the person selling the tools makes the most and in this case I think it continue to be cloud providers. I think in a very real way OpenAI and Anthropic are commercialized charities driving change and commoditizing rapidly their own products and it’ll be infrastructure providers who win the high end model game. I don’t think this is a problem I think this is in fact inline with their original charters but a different path than most people view nonprofit work. A much more capitalist and accelerated take.

Where they might make future businesses is in the tooling. My understanding from friends within these companies is their tooling is remarkably advanced vs generally available tech. But base models aren’t the future of revenues (to be clear tho they make considerable revenue today but at some point their efficiency will cannibalize demand and the residual business will be tools)

MP_172913• 2 years ago

I'm curious now. Can you give color on what you're doing that you keep hitting boundaries? I suppose it isn't limited by human-attention.

w10-1• 2 years ago

Since GPT-4 launch, token prices are falling 84% per year!! Good for mankind, but crazy for these companies

The message to competitor investors is that they will not make their money back.

OpenAI has the lead, in market and mindshare; it just has to keep it.

Competitors should realize they're better served by working with OpenAI than by trying to replace it - Hence the Apple deal.

Soon model construction itself will not be about public architectures or access to CPU's, but a kind of proprietary black magic. No one will pay for upstart 97% when they can get reliable 98% at the same price, so OpenAI's position will be secure.

abrichr• 2 years ago

where are my agents?

https://github.com/OpenAdaptAI/OpenAdapt/

golol9• 2 years ago

GPT-2: February 2019

GPT-3: June 2020

GPT-3.5: November 2022

GPT-4: March 2023

There were 3 years between GPT-3 and GPT-4!

whimsicalism14• 2 years ago

hardly anybody you are talking to even knows what gpt3 is, the time between 3.5 and 4 is what is relevant

MP_172913• 2 years ago

Obviously, I know these timetables.

But there's a light and day difference post-Nov22 than before. Both in the AI race it sparkled, but also in the funding all AI labs have.

If you're expecting GPT-5 by 2026, that's ok. Just very weird to me.

ugh123• 2 years ago

How people seriously expect these foundational model companies to make substantial revenue?

My take on this common question is that we haven't even begun to realize the immense scale of which we will need AI in all sorts of products, from consumer to enterprise. We will look back on the cost of tokens now (even at 50% of price a year or so ago) and look at it with the same bewilderment of "having a computer in your pocket" compared to mainframes from 50 years ago.

For AI to be truly useful at the consumer level, we'll need specialized mobile hardware that operates on a far greater scale of tokens and speed than anything we're seeing/trying now.

Think "always-on AI" rather than "on-demand".

siscia• 2 years ago

Now a bit of Shameless plug, but of you need an AI to take over your emails then my https://getgabrielai.com should cover most use cases.

* Summarisation * Smart filtering * Smart automatic drafting of replies

Very much in beta, and summarisation is still behind feature flag, but feel free to give it a try.

For summarisation here I mean to get one email with all your unread emails summarised.

joshstrange3• 2 years ago

Looking forward to trying this via ChatGPT. As always OpenAI says "now available" but refreshing or logging in/out of ChatGPT (web and mobile) don't cause GPT-4o to show up. I don't know why I find this so frustrating. Probably because they don't say "rolling out" they say things like "try it now" but I can't even though I'm a paying customer. Oh well...

glenstein4• 2 years ago

I think it's a legitimate point. For my personal use case, what are the most helpful things about these HN threads is comparing with others to see how soon I can expect it to be available for me. Like you, I currently don't have access, but I understand that it's supposed to become increasingly available throughout the day.

That is the text-based version. The full multimodal version I understand to be rolling out in the coming weeks.

candiodari• 2 years ago

I wonder if the audio stuff works like ViTS. Do they just encode the audio as tokens and input the whole thing? Wouldn't that make the context size a lot smaller?

One does notice that context size is noticeably absent from the announcement ...

cs7023• 2 years ago

The usual critics will quickly point out that LLMs like GPT-4o still have a lot of failure modes and suffer from issues that remain unresolved. They will point out that we're reaping diminishing returns from Transformers. They will question the absence of a "GPT-5" model. And so on -- blah, blah, blah, stochastic parrots, blah, blah, blah.

Ignore the critics. Watch the demos. Play with it.

This stuff feels magical. Magical. It makes the movie "Her" look like it's no longer in the realm of science fiction but in the realm of incremental product development. HAL's unemotional monotone in Kubrick's movie, "Space Odyssey," feels... oddly primitive by comparison. I'm impressed at how well this works.

Well-deserved congratulations to everyone at OpenAI!

CamperBob26• 2 years ago

Imagine what an unfettered model would be like. 'Ex Machina' would no longer be a software-engineering problem, but just another exercise in mechanical and electrical engineering.

The future is indeed here... and it is, indeed, not equitably distributed.

aftbit3• 2 years ago

Or from Zones of Thought series, Applied Theology, the study of communication with and creation of superhuman intelligences that might as well be gods.

aftbit3• 2 years ago

Who cares? This stuff feels magical. Magical!

On one hand, I agree - we shouldn't diminish the very real capabilities of these models with tech skepticism. On the other hand, I disagree - I believe this approach is unlikely to lead to human-level AGI.

Like so many things, the truth probably lies somewhere between the skeptical naysayers and the breathless fanboys.

CamperBob26• 2 years ago

On the other hand, I disagree - I believe this approach is unlikely to lead to human-level AGI.

You might not be fooled by a conversation with an agent like the one in the promo video, but you'd probably agree that somewhere around 80% of people could be. At what percentage would you say that it's good enough to be "human-level?"

micromacrofoot2• 2 years ago

I'm not so sure, I think this is what's called "emergent behavior" — we've found very interesting side effects of bringing together technologies. This might ultimately teach us more about intelligence than more reductionist approaches like scanning and mapping the brain.

layer87• 2 years ago

HAL's unemotional monotone in Kubrick's movie, "Space Odyssey," feels... primitive by comparison.

I’d strongly prefer that though, along with HAL’s reasoning abilities.

moffkalast6• 2 years ago

I would say a machine that thinks it feels emotions is less likely to throw you out of a spaceship. Human empathy already feels lacking compared to what something as basic as llama-3 can do.

jll292• 2 years ago

HAL has to sound exactly how Kubrick made it sound for the movie to work the way it should.

There wasn't any incentive to make it sound artificially emotional or emphatic beyond a "Sorry, Dave".

dragonwriter7• 2 years ago

This stuff feels magical. Magical.

Because its capacities are focused on exactly the right place to feel magical. Which isn’t to say that there isn’t real utility, but language (written, and even moreso spoken) has an enormous emotional resonance for humans, so this is laser-targeted in an area where every advance is going to “feel magical” whether or not it moves the needle much on practical utility; it’s not unlike the effect of TV news making you feel informed, even though time spent watching it negatively correlates with understanding of current events.

BoorishBears6• 2 years ago

You really think OpenAI has researchers figuring out how to drive emergent capabilities based on what markets well?

Edit: Apparently not based on your clarification, instead the researchers don't know any better than to march into a local maxima because they're only human and seek to replicate themselves. I assumed too much good faith.

aantix3• 2 years ago

Louis CK - Everything is amazing & nobody is happy

https://www.youtube.com/watch?v=kBLkX2VaQs4

agumonkey2• 2 years ago

I didn't use it as a textual interface, but as a relational/nondirectional system, trying to ask it to inverse recursive relationships (first/follow sets for BNF grammars). The fact that it could manage to give partially correct answers on such an abstract problem was "coldly" surprising.

DarkNova6• 2 years ago

VC loves it.

Another step closer for those 7 trillion that OpenAI is so desperate for.

ChuckMcM2• 2 years ago

Kind of this. That was one of the themes of the movie Westworld where the AI in the robots seemed magical until it was creepy.

I worry about the 'cheery intern' response becoming something of a punch line.

"Hey siri, launch the nuclear missiles to end the world."

"That's a GREAT idea, I'll get right on that! Is there anything else I can help you with?"

Kind of punch lines.

Will be interesting to see where that goes once you've got a good handle on capturing the part of speech that isn't "words" so much as it is inflection and delivery. I am interested in a speech model that can differentiate between "I would hate to have something happen to this store." as a compliment coming from a customer and as a threat coming from an extortionist.

cs7023• 2 years ago

Yes, the announcement explicitly states that much of the effort for this release was focused on things that make it feel magical (response times, multiple domains, etc.), not on moving the needle on quantifiable practical performance. For future releases, the clever folks at OpenAI are surely focused on improving performance on challenging tasks that practical utility -- while maintaining the "magical feeling."

benreesman• 2 years ago

It’s not an either-or: the stuff feels magical because it both represents dramatic revelation of capability and because it is heavily optimized to make humans engage in magical thinking.

These things are amazing compared to old-school NLP: the step-change in capability is real.

But we should also keep our wits about us, they are well-Des robed by current or conjectural mathematics, they fail at things dolphins can do, it’s not some AI god and it’s not self-improving.

Let’s have balance on both the magic of the experience and getting past the tech demo stage: every magic trick has a pledge, but I think we’re still working on the prestige.

porphyra• 2 years ago

Pretty interesting how it turns out that --- contrary to science fiction movies --- talking naturally and modelling language is much easier and was achieved much sooner than solving complex problems or whatever it is that robots in science fiction movies do.

xkcd-sucks• 2 years ago

its capacities are focused on exactly the right place to feel magical.

this focus subverts its intended effect on those of us with hair trigger bullshit-PTSD

agumonkey2• 2 years ago

That's what openai managed to catch. The large enough sense of wonder. You could feel it as people spread the news but not as the usual fad.. there was a soft silence to it, people focused deeply poking at it because it was a new interface.

barrell5• 2 years ago

Did you use any of the GPT voice features before? I’m curious whether this reaction is to the modality or the model.

Don’t get me wrong, excited about this update, but I’m struggling to see what is so magical about it. Then again, I’ve been using GPT voice every day for months, so if you’re just blown away from talking to a computer then I get it

famouswaffles24• 2 years ago

Speech is a lot more than just the words being conveyed.

Tone, Emphasis, Speed, Accent are all very important parts of how humans communicate verbally.

Before today, voice mode was strictly your audio>text then text>audio. All that information destroyed.

Now the same model takes in audio tokens and spits back out audio tokens directly.

Watch this demo, it's the best example of the kind of thing that would be flat out impossible with the previous setup.

https://www.youtube.com/live/DQacCB9tDaw?si=2LzQwlS8FHfot7Jy

mlsu2• 2 years ago

The voice modality plays a huge role in how impressive it seems.

When GPT-2/3/3.5/4 came out, it was fairly easy to see the progression from reading model outputs that it was just getting better and better at text. Which was pretty amazing but in a very intellectual way, since reading is typically a very "intellectual" "front-brain" type of activity.

But this voice stuff really does make it much more emotional. I don't know about you, but the first time I used GPT's voice mode I notice that I felt something -- very un-intellectually, very un-cerebral -- like, the feeling that there is a spirit embodying the computer. Of course with LLM's there always is a spirit embodying the computer (or, there never is, depending on your philosophical beliefs).

The Suno demos that popped up recently should have clued us all in that this kind of emotional range was possible with these models. This announcement is not so much a step function in model capabilities, but it is a step function in HCI. People are just not used to their interactions with a computer be emotional like this. I'm excited and concerned in equal parts that many people won't be truly prepared for what is coming. It's on the horizon, having an AI companion, that really truly makes you feel things.

Us nerds who habitually read text have had that since roughly GPT-3, but now the door has been blown open.

rrrrrrrrrrrryan2• 2 years ago

Yeah the product itself is only incrementally better (lower latency responses + can look at a camera feed, both great improvements but nothing mindblowing or "magical"), but I think the big difference is that this thing is available for free users now.

grantsucceeded2• 2 years ago

Magical?

the interruptiopn part is just flow control at the edge. control-s, control-c stuff, right? not AI?

The sound of a female voice to an audience 85% composed of males between the ages of 14 and 55 is "magical", not this thing that recreates it.

so yeah, its flow control and compression of highly curated, subtle soft porn. Subtle, hyper targeted, subconscious porn honed by the most colossal digitally mediated focus group ever constructed to manipulate our (straight male) emotions.

why isn't the voice actually the voice of the pissed off high school janitor telling you to man-up and stop hyperventilating? instead its a woman stroking your ego and telling you to relax and take deep breaths. what dataset did they train that voice on anyway?

mindcrime• 2 years ago

I may or may not entirely agree with this sentiment (but I definitely don't disagree with all of it!) but I will say this: I don't think you deserve to be downvoted for this. Have a "corrective upvote" on me.

whimsicalism14• 2 years ago

Right, because having a female voice means that it is soft porn.

This is like horseshoe theory on steroids.

micromacrofoot2• 2 years ago

It's not that complicated, generally more woman-like voices test as more pleasant to men and women alike. This concept has been backed up by stereotypes for centuries.

Most voice assistants have male options, and an increasing number (including ChatGPT) have gender neutral voices.

why isn't the voice actually the voice of the pissed off high school janitor telling you to man-up and stop hyperventilating

sounds like a great way to create a product people will outright hate

Melatonic• 2 years ago

HAL's voice acting I would say is actually superb and super subtly very much not unemotional. Part of what makes so unnerving. They perfect nailed creepy uncanny valley

WhitneyLand2• 2 years ago

How much of this could be implemented using the API?

There’s so much helpful niche functionality that can be added to custom clients.

OOPMan• 2 years ago

I really don't think Sam needs more encouragement, thanks.

Also, if this is your definition of magic then...yeah...

noman-land• 2 years ago

Magic is maybe not the best analogy to use because magic itself isn't magical. It is trickery.

karmasimida2• 2 years ago

Very convincing demo

However, using ChatGPT with transcribing is already offering me similar experience, so what is new exactly

scarface_744• 2 years ago

Some of the failure modes in LLMs have been fixed by augmenting LLMs with external services

The simplest example is “list all of the presidents in reverse chronological order of their ages when inaugurated”.

Both ChatGpt 3.5 and 4 get the order wrong. The difference is that I can instruct ChatGPT 4 to “use Python”

https://chat.openai.com/share/87e4d37c-ec5d-4cda-921c-b6a9c7...

You can do similar things to have it verify information by using internet sources and give you citations.

Just like with the Python example, at least I can look at the script/web citation myself

wintermutestwin• 2 years ago

It is pretty awesome that you only have to prompt with “use python”

aspenmayer5• 2 years ago

The simplest example is “list all of the presidents in reverse chronological order of their ages when inaugurated”.

This question is probably not the simplest form of the query you intend to receive an answer for.

If you want a descending list of presidents based on their age at inauguration, I know what you want.

If you want a reverse chronological list of presidents, I know what you want.

When you combine/concatenate the two as you have above, I have no idea what you want, nor do I have any way of checking my work if I assume what you want. I know enough about word problems and how people ask questions to know that you probably have a fairly good idea what you want and likely don’t know how ambitious this question is as asked, and I think you and I both are approaching the question with reasonably good faith, so I think you’d understand or at least accommodate my request for clarification and refinement of the question so that it’s less ambiguous.

Can you think of a better way to ask the question?

Now that you’ve refined the question, do LLMs give you the answers you expect more frequently than before?

Do you think LLMs would be able to ask you for clarification in these terms? That capability to ask for clarification is probably going to be as important as other improvements to the LLM, for questions like these that have many possibly correct answers or different interpretations.

Does that make sense? What do you think?

croes2• 2 years ago

This stuff feels magical. Magical.

Sound like the people who defend Astrology because it feels magical how their horoscope fits their personality.

"Don't bother me with facts that destroy my rose-tinted view"

At moment AI is a massive hype and shoved into everything. To point at the faults and weaknesses is a reasonable and responsible thing to do.

hsavit1• 2 years ago

yea, we don't want or need this kind of "magic" - because it's hardly magic to begin with, and it's more socially and environmentally destructive than anything else.

helicalmix10• 2 years ago

i legitimately don't understand this viewpoint.

3 years ago, if you told me you could facetime with a robot, and they could describe the environment and have a "normal" conversation with me, i would be in disbelief, and assume that tech was a decade or two in the future. Even the stuff that was happening a 2 years ago felt unrealistic.

astrology is giving vague predictions like "you will be happy today". GPT-4o is describing to you actual events in real time.

whimsicalism14• 2 years ago

Sound like the people who defend Astrology because it feels magical how their horoscope fits their personality.

Does it really or are you just playing facile word association games with the word "magical"?

arisAlexis• 2 years ago

What is the point of pointing faults that will be fixed very soon? Just being negative or unable to see the future?

idopmstuff4• 2 years ago

Astrology is a thing with no substance whatsoever. It's just random, made-up stories. There is no possibility that it will ever develop into something that has substance.

AI has a great deal of substance. It can draft documents. It can identify foods in a picture and give me a recipe that uses them. It can create songs, images and video.

AI, of course, has a lot of flaws. It does some thing poorly, it does other things with bias, and it's not suitable for a huge number of use cases. To imply that something that has a great deal of substance but flaws alongside is the same as something that has no substance whatsoever nor ever will is just not a reasonable thing to do.

dogcomplex• 2 years ago

If you want to talk facts, then those critics are similarly on weak grounds and critiquing feelings more than facts. There has been no actual sign of scaling ceasing to work, in medium after medium, and most of their criticisms are issues with how LLM tools are embedded in architectures which are still incredibly early/primitive and still refining how to use transformers effectively. We haven't even begun using error correction techniques from analog engineering disciplines properly to boost the signal of LLMs in practical settings. There is so much work to do with just the existing tools.

"AI is massive hype and shoved into everything" has more grounding as a negative feeling of people being overwhelmed with technology than any basis in fact. The faults and weaknesses are buoyed by people trying to acknowledge your feelings than any real criticism of a technology that is changing faster than the faults and weakness arguments can be made. Study machine learning and come back with an informed criticism.

pmelendez• 2 years ago

Ignore the critics. Watch the demos. Play with it

With so many smoke and mirrors demos out there, I am not super excited at those videos. I would play with it, but it seems like it is not available in a free tier (I stopped paying OpenAI a while ago after realizing that open models are more than enough for me)

m463• 2 years ago

HAL's unemotional monotone

on a tangent...

I find it interesting the psychology behind this. If the voice in 2001 had proper inflection, it wouldn't have been perceived as a computer.

(also, I remember when voice synthesizers got more sophisticated and Stephen Hawking decided to keep his original first-gen voice because he identified more with it)

I think we'll be going the other way soon. Perfect voices, with the perfect emotional inflection will be perceived as computers.

However I think at some point they may be anthropomorphized and given more credit than they deserve. This will probably be cleverly planned and a/b tested. And then that perfect voice, for you, will get you to give in.

0xdeadbeefbabe• 2 years ago

HAL's unemotional monotone in Kubrick's movie, "Space Odyssey," feels... oddly primitive by comparison

In comparison to the gas pump which says "Thank You!"

nojvek4• 2 years ago

Play with it!

It’s not accessible to everyone yet.

Even on api, I can’t send it voice stream yet.

Api refuses to generate images.

Next few weeks will tell as more people play with it.

fhub• 2 years ago

I prompted it with "Take this SSML script and give me a woman's voice reading it as WAV or MP3 [Pasted script]" and it pretty much sounds like HAL.

speedgoose2• 2 years ago

Did they release the new voices yet?

password54321• 2 years ago

Comments have become insufferable. Either it is now positive to the point of bordering on cringe-worthiness (your comment) or negative. Nuanced discussion is dead.

smugglerFlynn• 2 years ago

Watching HAL happening in real life comes across as creepy, not magical. Double creepy with all the people praising this ‘magicality’.

I’m not a sceptic and apply AI on a daily basis, but whole “we can finally replace people” vibe is extremely off-putting. I had very similar feelings during pandemic, when majority of people was so seemingly happy to drop any real human interaction in favor of remote comms via chats/audio calls, it still creeps me out how ready we are as a society to drop anything remotely human in favor of technocratic advancement and “productivity”.

aiauthoritydev• 2 years ago

1. Demos are meant for feel magical and except in Apple's case they are often exaggerated versions of their real product.

2. Even then this is a wonderful step for tech in general and not just OpenAI. Makes me very excited.

3. Most economic value and growth driven by AI will not come from consumer apps but rather the enterprise use. I am interested in seeing how AI can automatically buy stuff for me, automate my home, reduce my energy used, automatically apply and get credit cards based on my purchases, find new jobs for me, negotiate with a car dealer on my behalf, detect when I am going to fall sick, better diabetes case and eventual cure etc. etc.

lm284695• 2 years ago

It makes the movie "Her" look like it's no longer in the realm of science fiction but in the realm of incremental product development

Are we supposed to cheer to that?

We're already mid way to the full implementation of 1984, do we need Her before we get to Matrix ?

throwthrowuknow14• 2 years ago

Her wasn’t a dystopia as far as I could tell. Not even a cautionary tale. The scifi ending seems unlikely but everything else is remarkably prescient. I think the picnic scene is very likely to come true in the near future. Things might even improve substantially if we all interact with personalities that are consistently positive and biased towards conflict resolution and non judgemental interactions.

bowsamic• 2 years ago

The demos seem quite boring to me

suarezluis• 2 years ago

This is such a hot take, it should go in hot-takes.io LOL

goatlover6• 2 years ago

It makes the movie "Her" look like it's no longer in the realm of science fiction but in the realm of incremental product development.

The last part of the movie "Her" is still in the realm of science fiction, if not outright fantasy. Reminds me of the later seasons of SG1 with all the talk of ascension and Ancients. Or Clarke's 3001 book intro, where the monolith creators figured out how to encode themselves into spacetime. There's nothing incremental about that.

badgersnake• 2 years ago

Blah blah blah indeed, the hype train continues unabated. The problem is, those are all perfectly valid criticisms and LLMS can never live up to the ridiculous levels of hype.

peterisza• 2 years ago

Can anybody help me try the direct voice feature? I can't find the button for it. Maybe it's not available in Europe yet, I don't know.

cess11• 2 years ago

You'll have a great time once you discover literature. Especially early modern novels, texts the authors sometimes spent decades refining, under the combined influences of classical arts and thinking, Enlightenment philosophy and science.

If chatbots feel magical, what those people did will feel divinely inspired.

vwkd• 2 years ago

Funnily, I’d prefer HAL’s unemotional monotone over GPT’s woke hyperbola any second.

byw• 2 years ago

I mean, humans also have tons of failures modes, but we've learned to live them over time.

The average human have tons of quirks, talk over each other all the time, generally can't solve complex problems in a casual conversion setting, and are not always cheery and ready to please like Scarlet's character in Her.

I think our expectations of AI is way too high from our exposure to science fiction.

bearjaws• 2 years ago

OAI just made an embarrassment of Google's fake demo earlier this year. Given how this was recorded, I am pretty certain it's authentic.

CivBase2• 2 years ago

I don't doubt this is authentic, but if they really wanted to fake those demos, it would be pretty easy to do using pre-recorded lines and staged interactions.

mike006324• 2 years ago

For what it's worth, OpenAI also shared videos of failed demos:

https://vimeo.com/945591584

I really value how open they are being about its limitations.

hehdhdjehehegwv4• 2 years ago

This feature has been in iOS for a while now, just really slow and without some of the new vision aspects. This seems like a version 2 for me.

bigyikes12• 2 years ago

That old feature uses Whisper to transcribe your voice to text, and then feeds the text into the GPT which generates a text response, and then some other model synthesizes audio from that text.

This new feature feeds your voice directly into the GPT and audio out of it. It’s amazing because now ChatGPT can truly communicate with you via audio instead of talking through transcripts.

New models should be able to understand and use tone, volume, and subtle cues when communicating.

I suppose to an end user it is just “version 2” but progress will become more apparent as the natural conversation abilities evolve.

abhpro• 2 years ago

No it's not the same thing, the link for this submission even explains that. Anyone who comments should at least give the submission a cursory read.

readams• 2 years ago

https://twitter.com/Google/status/1790055114272612771

ianbicking5• 2 years ago

This demo feels a lot like GPT-V. Like they've gotten a lot of the latencies down, but it's doing the same thing GPT was doing previously with transcription after silence detection and TTS of the output.

sumedh4• 2 years ago

Is there a reason why Open AI and Google have events so close to each other?

nojvek4• 2 years ago

Let OAI actually be released to the masses. Then we can compare.

I’m not a big fan of announcing something but it not being released.

They say available for api but it’s text only. Can’t send audio stream to get audio stream back.

Time will tell. I’m holding my emotions after I get my hands on it.

levocardia• 2 years ago

As a paid user this felt like a huge letdown. GPT-4o is available to everyone so I'm paying $20/mo for...what, exactly? Higher message limits? I have no idea if I'm close to the message limits currently (nor do I even know what they are). So I guess I'll cancel, then see if I hit the limits?

I'm also extremely worried that this is a harbinger of the enshittification of ChatGPT. Processing video and audio for all ~200 million users is going to be extravagantly expensive, so my only conclusion is that OpenAI is funding this by doubling down on payola-style corporate partnerships that will result in ChatGPT slyly trying to mention certain brands or products in our conversations [1].

I use ChatGPT every day. I love it. But after watching the video I can't help but think "why should I keep paying money for this?"

[1] https://www.adweek.com/media/openai-preferred-publisher-prog...

muttantt• 2 years ago

So... cancel the subscription?

CodeCrusader• 2 years ago

Completely agree, none of the updates will apply to any of my use cases, disappointment.

noncoml• 2 years ago

They really need to tone down the talking garniture. It needs to put on its running shoes and get to the point on every reply. Ain’t nobody has time to keep listening to AI blubbering along at every prompt.

dbcooper• 2 years ago

question for you guys - is there a model that can take figures (graphs), from scientific publications, and combine image analysis with picking up the data point symbol descriptions and analyse the trends?

krunck• 2 years ago

So GPT-4o can do voice intonation? Great. Nice work.

Still, it sounds like some PR drone selling a product. Oh wait....

CivBase2• 2 years ago

Those voice demos are cool but having to listen to it speak makes me even more frustrated with how these LLMs will drone on and on without having much to say.

For example, in the second video the guy explains how he will have it talk to another "AI" to get information. Instead of just responding with "Okay, I understand" it started talking about how interesting the idea sounded. And as the demo went on, both "AIs" kept adding unnecessary commentary about the secenes.

I would hate having to talk with these things on a regular basis.

golol9• 2 years ago

Yea at some pont the style and tone of these assistants needs to be seriously changed, I can imagine a lot of their RLHF and instruct processes emphasize sounding good vs being good too much.

DataDaemon• 2 years ago

Now, say goodbye to call centers.

willsmith722• 2 years ago

and say hello to your grandma getting scammed

gverrilla4• 2 years ago

and say hello to UBI which in the long term will diminish scam :)

joshstrange3• 2 years ago

What do they mean by "desktop version"? I assume that doesn't mean a "native" (electron) app?

simonw11• 2 years ago

I'm seeing gpt-4o in the OpenAI Playground interface already: https://platform.openai.com/playground/chat?mode=chat&model=...

First impressions are that it feels very fast.

tailspin20192• 2 years ago

Does anyone with a paid plan see anything different in the ChatGPT iOS app yet?

Mine just continues to show “GPT 4” as the model - it’s not clear if that’s now 4o or there is an app update coming…

ilaksh3• 2 years ago

Are there any remotely comparable open source models? Fully multimodal, audio-to-audio?

MBCook2• 2 years ago

Too bad they consume 25x the electricity Google does.

https://www.brusselstimes.com/world-all-news/1042696/chatgpt...

simonw11• 2 years ago

That's not a well sourced story: it doesn't say where the numbers come from. Also:

"However, ChatGPT consumes a lot of energy in the process, up to 25 times more than a Google search."

That's comparing a Large Language Model prompt to a search query.

joshstrange3• 2 years ago

Too bad they consume 25x the electricity Google does.

From the article:

"However, ChatGPT consumes a lot of energy in the process, up to 25 times more than a Google search."

And the article doesn't back that claim up nor do they break out how much energy ChatGPT (A Message? Whole conversation? What?) or a Google search uses. Honestly the whole article seems very alarmist while being light on details and making sweeping generalizations.

rvnx• 2 years ago

And in this 25x you get your answer.

What if we actually counted the electricity that the websites use instead of just the search engine page ?

delichon• 2 years ago

Won't this make pretty much all of the work to make a website accessible go away, as it becomes cheap enough? Why struggle to build alt content for the impaired when it can be generated just in time as needed?

And much the same for internationalization.

unix_fan• 2 years ago

That’s just one part of the issue. The other part of the issue is accessibility bugs, you would have to get the model to learn to use a screen reader, and then change things as needed

asadotzler2• 2 years ago

Because accessibility is more than checking a box. Got a photo you took on your website? Alternative text that you wrote capturing what you see in the photo you took is accessibility done right. Alt text generated by bots is not accessibility done right unless that bot knows what you see in the photo you took and that's not likely to happen.

Negitivefrags• 2 years ago

I found these videos quite hard to watch. There is a level of cringe that I found a bit unpleasant.

It’s like some kind of uncanny valley of human interaction that I don’t get on nearly the same level with the text version.

jameshart12• 2 years ago

While it is probably pretty normal for California, the insincere flattery and patronizing eagerness are definitely grating But then you have to stack that up against the fact that we are examining a technology and nitpicking over its tone of voice.

MattPalmer10863• 2 years ago

I found it disturbing that it had any kind of personality. I don't want a machine to pretend to be a person. I guess it makes it more evident with a voice than text.

But yeah, I'm sure all those things would be tunable, and everyone could pick their own style.

zamadatix3• 2 years ago

I feel like it's largely an effect of tuning it to default as "a ultra helpful assistant which is happy to help with any request via detailed responses in candid and polite manner..." kind of thing as you basically lose free points any time it doesn't jump on helping with something, tries to use short output and generates a more incorrect answer as a result, or just plain has to be initialized with any of this info.

It seems like both the voice and responses can be tuned pretty easily though so hopefully that kind of thing can just be loaded in your custom instructions.

TaylorAlexander3• 2 years ago

I’m born and raised in California and I think I’m a pretty “California” person (for better and worse).

It feels exhausting watching these demos and I’m not excited at all to try it. I really don’t feel the need for an AI assistant or chatbot to pretend to be human like this. It just feels like it’s taking longer to get the information I want.

You know in the TV series “Westworld” they have this mode, called “analysis”, where they can tell the robots to “turn off your emotional affect”.

I’d really like to see this one have that option. Hopefully it will comply if you tell it, but considering how strong some of the RLHF has been in the past I’m not confident in that.

652• 2 years ago

Maybe they need to make a North Easterner mode. Cut the bullshit and pleasantries. Give me the information and move on!

MattPalmer10863• 2 years ago

I had the same reaction. While incredibly impressive, it wasn't something I would want to interact with.

selfmodruntime9• 2 years ago

Yes. This model - and past models to an extent - have a very unique american and californian feel to them in their response. I am German for example, and day to that conversations lack any superficial flattery so much that the demo feels extreme to me.

elliotto• 2 years ago

All these demo style ads/videos are super jarring and uncanny valley-esque to watch as an Australian. The US corporate cultural norms are super bizarre to the rest of the world, and the California based holy omega of tech companies really takes this to the extreme. The application might work well if you interact with it like you are a normal human being - but I can't tell because this presentation is corporate robots talking to machine robots.

HHC-Hunter• 2 years ago

Agreed. Americans why are you like this?

prawn3• 2 years ago

That was my reaction (as an Australian) too. The AI is so verbose and chirpy by default. There was even a bit in one video where he started talking over the top of the AI because it was rabbiting on.

But I find the text version similar. Delivers too much and too slowly. Just get me the key info!

clhodapp3• 2 years ago

It's like Project Milo grew up and became a white color worker.

jameshart12• 2 years ago

I know you meant to type 'collar', but your typo's not wrong...

bambax2• 2 years ago

Talking with people is hard enough. I need to see the people I'm talking to, or I'd rather write, because it's asynchronous and I have all the time I need to organize my message.

I think all the fakery in those demos help in that regard: it narrows the field of the possible interpretations of what is being said.

ByThyGrace• 2 years ago

The what I presume default female voice and tone speaks like an AI character straight out of a Black Mirror episode.

saxelsen• 2 years ago

The demo where they take turns singing felt like two nervous slaves trying to please their overlord who kept interrupting them and demanding more harmony.

dyauspitr11• 2 years ago

It is that and that’s okay, they’re algorithms.

xfz• 2 years ago

Nauseating mode is the default, you'll have to pay extra for a tolerable personality. ;)

Seriously though, I'm sure it's an improvement but having used the existing voice chat I think they had a few things to address. (Perhaps 4o does in some cases).

- Unlike the text interface it asks questions to keep the conversation going. It feels odd when I already got the answer I wanted. Clarifying questions yes, pretending to be a buddy - I didn't say I was lonely, I just asked a question! It makes me feel pressured to continue.

- Too much waffle by far. Give me short answers, I am capable of asking follow up questions.

- Unable to cope with the mechanics of usual conversation. Pausing before adding more, interrupting, another person speaking.

- Only has a US accent, which is fine but not what I expect when Google and Alexa have used British English for many years.

Perhaps they've overblown the "personality" to mask some of these deficiencies?

Not saying it's easy to overcome all the above but I'd rather they just dial down the intonation in the meantime.

borgdefense4• 2 years ago

I love the Juniper voice in chatGPT4.

I am blown away having spent hours prompting GPT4o.

If it can give shorter answers in voice mode instead of lectures then a back and forth conversation with this much power can be quite interesting.

I still doubt I would use it that much though just because of how much is lost compared to the screen. Code and voice make no sense. The time between prompts usually requires quite a bit of thought for anything interesting that a conversation itself is only useful for things I have already asked it.

For me, gpt4 is already as useless as 3.5. I will never prompt gpt4 again. I can still push GPT4o over the edge in python but damn, it is pretty out there. Then the speed is really amazing.

brainer• 2 years ago

OpenAI's Mission and the New Voice Mode of GPT-4

• Sam Altman, the CEO of OpenAI, emphasizes two key points from their recent announcement. Firstly, he highlights their commitment to providing free access to powerful AI tools, such as ChatGPT, without advertisements or restrictions. This aligns with their initial vision of creating AI for the benefit of the world, allowing others to build amazing things using their technology. While OpenAI plans to explore commercial opportunities, they aim to continue offering outstanding AI services to billions of people at no cost.

• Secondly, Altman introduces the new voice and video mode of GPT-4, describing it as the best compute interface he has ever experienced. He expresses surprise at the reality of this technology, which provides human-level response times and expressiveness. This advancement marks a significant change from the original ChatGPT and feels fast, smart, fun, natural, and helpful. Altman envisions a future where computers can do much more than before, with the integration of personalization, access to user information, and the ability to take actions on behalf of users.

https://blog.samaltman.com/gpt-4o

simonw11• 2 years ago

Please don't post AI-generated summaries here.

reisse2• 2 years ago

The facts that AI-generated summaries are still detected instantaneously and are bad enough for people to explicitly ask not to post them says something about current state of LLMs.

bamboozled13• 2 years ago

access to user information,

Sam, please stop ok, those things you saw on tv when you were a kid? They were dystopian movies, we don’t want that for real, ok?

deegles• 2 years ago

what's the path from LLMs to "true" general AI? is it "only" more training power/data or will they need a fundamental shift in architecture?

banjoe• 2 years ago

I still need to talk very fast to actually chat with ChatGPT which is annoying. You can tell they didn't fix this based on how fast they are talking in the demo.

gallerdude• 2 years ago

Interesting that they didn't mention a bump in capabilities - I wrote a LLM benchmark a few weeks ago, and before GPT-4 could solve Wordle about ~48% of the time.

Currently with GPT-4o, it's easily clearing 60% - while blazing fast, and half the cost. Amazing.

dom96• 2 years ago

I can't help but feel a bit let down. The demos felt pretty cherry picked and still had issues with the voice getting cut off frequently (especially in the first demo).

I've already played with the vision API, so that doesn't seem all that new. But I agree it is impressive.

That said, watching back a Windows Vista speech recognition demo[1] I'm starting to wonder if this stuff won't have the same fate in a few years.

1 - https://www.youtube.com/watch?v=VMk8J8DElvA

quenix• 2 years ago

I think the voice was getting cut off because it heard the crowd reaction and paused (basically it's a feature, not a bug).

kthartic3• 2 years ago

The voice getting cut off was likely just a problem with their live presentation setup, not the ChatGPT app. It was flawless in the 2nd half of the presentation.

jrflowers• 2 years ago

I like the robot typing at the keyboard that has B as half of the keys and my favorite part is when it tears up the paper and behind it is another copy of that same paper

hu3• 2 years ago

That they are offering more features for free concurs with my theory that, just like search, state of the art AI will soon be "free", in exchange for personal information/ads.

martingalex2• 2 years ago

Need more data.

BurkeTarget• 2 years ago

People are directly asking for suggestion/recommendations on products or places. They'll more than recoup the costs by selling top rank on those questions.

CosmicShadow• 2 years ago

In the video where the 2 AI's sing together, it starts to get really cringey and weird to the point where it literally sounds like it's being faked by 2 voice actors off-screen with literal guns to their heads trying not to cry, did anyone else get that impression?

The tonal talking was impressive, but man that part was like, is someone being tortured or forced against their will?

flakiness• 2 years ago

Here is the link: https://www.youtube.com/watch?v=Bb4HkLzatb4

I think this demo is more for showing the limit like "It can sing isn't it amazing?" than being practical, and I think it perfectly served the purpose.

I agree that the tortured impression. It partly comes from the facial expression of the presenter. She's clearly enjoying pushing it to the edge.

bigyikes12• 2 years ago

It didn’t just demonstrate the ability to sing, but also the ability for two AIs to cooperate! I’m not sure which was more impressive

zulban• 2 years ago

Absolutely. It felt like some kind of new uncanny valley. The over the top happiness of the forced singing sounded like torture, or like they were about to cry.

Amazing tech, but that was my human experience of it.

mickg102• 2 years ago

So, babelfish soon?

taytus2• 2 years ago

the OpenAI live stream was quite underwhelming...

mickg102• 2 years ago

So, babelfish incoming?

alvaroir• 2 years ago

I'm really impressed about this demo! Apart from the usual quality benchmarks I'm really impressed about the latency for audio/video: "It can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response"... If true at scale, what could be the "tricks" they're using for achieving that?!

Thaxll• 2 years ago

It's pretty impressive, although I don't like the voice / tone, I prefer something more neutral.

blixt2• 2 years ago

GPT-4o being a truly multimodal model is exciting, does open the door to more interesting products. I was curious about the new tokenizer which uses much fewer tokens for non-English, but also 1.1x fewer tokens for English, so I'm wondering if this means each token now can be more possible values than before? Might make sense provided that they now also have audio and image output tokens? https://openai.com/index/hello-gpt-4o/

I wonder what "fewer tokens" really means then, without context on raising the size of each token? It's a bit like saying my JPEG image is now using 2x fewer words after I switched from a 32-bit to a 64-bit architecture no?

zackangelo• 2 years ago

New tokenizer has a much larger vocabulary (200k)[0].

[0] https://github.com/openai/tiktoken/commit/9d01e5670ff50eb74c...

bigyikes12• 2 years ago

Besides increasing the vocabulary size, one way to use “fewer tokens” for a given task is to adjust how the tokenizer is trained with respect to that task.

If you increase the amount of non-English language representation in your data set, there will be more tokens which cover non-English concepts.

The previous tokenizer infamously required many more tokens to express a given concept in Japanese compared to English. This is likely because the data the tokenizer was trained on (which is not necessarily the same data the GPT model is trained on) had a lot more English data.

Presumably the new tokenizer was trained on data with a higher proportion of foreign language use and lower proportion of non-language use.

kolinko3• 2 years ago

The size can stay the same. Tokens get converted into state which is a vector of 4000+ dimensions. So you could have millions of tokens even and still encode them into the same state size.

catchnear4321• 2 years ago

window dressing

his love for yud is showing.

frabcus2• 2 years ago

I can't see any calculator for the audio pricing (https://openai.com/api/pricing/) or document type field in the Chat Completions API (https://platform.openai.com/docs/api-reference/chat/create) for this new model.

Is the audio in API not available yet?

willsmith722• 2 years ago

We plan to launch support for GPT-4o's new audio and video capabilities to a small group of trusted partners in the API in the coming weeks.

So no word on an audio api for regular joes? that's the number 1 thing i'm looking for

UncleOxidant• 2 years ago

Looking at the demo video, the AIs are a bit too chatty. The human has to often interrupt them.

A nice feature would be to be able to select a Meyer's Briggs personality type for your AI chatbot.

michalf6• 2 years ago

I cannot find the mac app anywhere. Is there a link?

Painsawman123• 2 years ago

My main takeaway is that Generative AI has hit a wall... New paradigms, architectures and breakthroughs are necessary for the field to progress but this begs the question, If everyone knows the current paradigms have hit a wall, Why is so much money being spent on LLMs ,diffusion models etc,which are bound to become obsolete within a few(?) years?

I_am_tiberius• 2 years ago

Interested in how many LLM startups there are that are going out of business due to this voice assistant.

windowshopping• 2 years ago

There's a button on this page that says "try on ChatGPT ->" but that's still version 3.5 and if I upgraded seems to be version 4.

Is this new version not available to users yet?

xyst• 2 years ago

The naming of these systems has me dead

nikolay• 2 years ago

I am a paid customer, yet I don't see anything new. I'm tired of these fake announcements of "released" features.

seydor2• 2 years ago

I would still prefer the features in text form, in the chat GUI. Right now chatGPT doesnt seem to have options to lengthen parts of the text response, to change it etc. Perplexity and gemini do seem to get the gui right. Voice chat is fun for demos but won't catch much, just like all the predecessors. Perhaps an advanced version of this could be used as a student tutor however

benlivengood2• 2 years ago

I am guessing text chat will be improved in all multimodal models because they have a broader base of data for pre-training. Benchmarks seem to show 4o slightly exceeding 4 (despite being a smaller model, or at least more parallelizable)

Satam• 2 years ago

So far OpenAI's template is: amazing demos create hype -> reality turns out to be underwhelming.

Sora is not yet released and not clear when it will be. Dall-e is worse than mid-journey in most cases. GPT-4 has either gotten worse or stayed the same. Vision is not really usable for anything practical. Voice is cool but not that useful, especially with lack of strong reasoning from the base model.

Is this sandbagging or is the progress slower than what they're broadcasting?

zone411• 2 years ago

It doesn't improve on NYT Connections leaderboard:

GPT-4 turbo (gpt-4-0125-preview) 31.0

GPT-4o 30.7

GPT-4 turbo (gpt-4-turbo-2024-04-09) 29.7

GPT-4 turbo (gpt-4-1106-preview) 28.8

Claude 3 Opus 27.3

GPT-4 (0613) 26.1

Llama 3 Instruct 70B 24.0

Gemini Pro 1.5 19.9

Mistral Large 17.7

gentile• 2 years ago

There is a spelling mistake in the japanese translation under language tokenization. In こんにちわ, わ should be は.

stilwelldotdev• 2 years ago

I love that there is a real competition happening. We're going to see some insane innovations.

buildbot• 2 years ago

The demo is impressive but I get very odd/cringe "Her" vibes from it as well.

ravroid• 2 years ago

In my experience so far, GPT-4o seems to sit somewhere between the capability of GPT-3.5 and GPT-4.

I'm working on an app that relies more on GPT-4's reasoning abilities than inference speed. For my use case, GPT-4o seems to do worse than GPT-4 Turbo on reasoning tasks. For me this seems like a step-up from GPT-3.5 but not from GPT-4 Turbo.

At half the cost and significantly faster inference speed, I'm sure this is a good tradeoff for other use cases though.

mike006324• 2 years ago

I have never tried GPT-4 because I don't pay for it. I'm really looking forward to GPT-4o being released to free tier users.

lwansbrough• 2 years ago

Very impressive. Please provide a voice that doesn't use radio jingle intonation, it is really obnoxious.

I'm only half joking when I say I want to hear a midwestern blue collar voice with zero tact.

tr3ntg4• 2 years ago

Copied and pasted the robot image journaling prompt and it simply cannot produce legible text. The first few words work, but the rest becomes gibberish. I wonder if there's weird prompt engineering squeezing out that capability or if its a 1 in a million chance.

ajdoingnothing• 2 years ago

If there was any glimmer of hope for "Rabbit M1" or "Humane AI pin", it can be buried to dust.

bossyTeacher2• 2 years ago

I will be the one to say it.

Progress is slowing down. Ever since gpt3, periods of time between releases are getting longer and the improvements are smaller. Your average non-techie investor is on the LLM hype train and is willing to dump a questionable amounts of money on LLM development. Who is going to explain to him/her/them that the LLM hype train is slowly losing steam?

Hopefully, before the LLM hype dies, another [insert here new ANN architecture], will bring better results than LLMs and another hype cycle will begin.

Every time we make a new breathrough, people think that the discovery rate is going to be linear or exponential when the beginning is closer to a logarithmic rate with the tail end resulting in diminishing returns

unglaublich• 2 years ago

I hope we can disable the cringe American hyperemotions.

stavros• 2 years ago

I made a website with book summaries (https://www.thesummarist.net/) and I tested GPT-4o in generating one, and it was bad. It reminded me of GPT-3.5. I didn't test too much, but preliminary results don't look good.

glenstein4• 2 years ago

Text access rolling out today, apparently:

GPT-4o’s text and image capabilities are starting to roll out today in ChatGPT. We are making GPT-4o available in the free tier, and to Plus users with up to 5x higher message limits.

Anyone have access yet? Not there for me so far.

toxic722• 2 years ago

It shows available for me in the OpenAI playground currently.

afro882• 2 years ago

I have access to GPT-4o text and audio, but no video. This is on the iOS app with ChatGPT Plus subscription.

Initial connection for audio fails most of the time, but once it's connected it's stable. Sometimes a bit more latency than expected, but mostly just like the demos.

famouswaffles24• 2 years ago

Is it really 4o audio? Some people still have the old audio while using 4o for text.

m3kw92• 2 years ago

The big news is that this is gonna be free

wesleyyue• 2 years ago

If anyone wants to try it for coding, I just added support for GPT4o in Double (https://double.bot)

In my tests:

* I have a private set of coding/reasoning tests and it's been able to ace all of them so far, beating Opus, GPT4-Turbo, and Llama 3 70b. I'll need to find even more challenging tests now...

* It's definitely significantly faster, but we'll see how much of this is due to model improvements vs over provisioned capacity. GPT4-Turbo was also significantly faster at launch.

loveiswork• 2 years ago

While I do feel a bit of "what is the point of my premium sub", I'm really excited for these changes.

Considering our brain is a "multi-modal self-reinforcing omnimodel", I think it makes sense for the OpenAI team to work on making more "senses" native to the model. Doing so early will set them up for success when future breakthroughs are made in greater intelligence, self-learning, etc.

652• 2 years ago

Time to bring back Luddism.

OutOfHere5• 2 years ago

I am observing an extremely high rate of text hallucinations with gpt-4o (gpt-4o-2024-05-13) as tested via the API. I advise extreme caution with it. In contrast, I see no such concern with gpt-4-turbo-preview (gpt-4-0125-preview).

fdb• 2 years ago

Same here. I observed it making up functions in d3 (`d3.geoProjectionRaw` and `d3.geoVisible`), in addition to ignoring functions it could have used.

bigyikes12• 2 years ago

If true, makes me wonder what kind of regression testing OpenAI does for these models. It can’t be easy to write a unit test for hallucinations.

OutOfHere5• 2 years ago

At a high level, ask it to produce a ToC of information about something that you know will exist in the future, but does not yet exist, but also tell it to decline the request if it doesn't verifiably know the answer.

Seattle35032• 2 years ago

I asked it to describe the books of an obscure author. GPT-4o hallucinated books. GPT-4 knew it needed to do an internet search.

mtam• 2 years ago

GPT-4o is very fast but seems to generate some very random ASCII Art compared to GPT-4 when text in the art is involved.

ta-run3• 2 years ago

This looks too good to be true? What's the catch?

Also, wasn't expecting the perf to improve by 2x

0xbadc0de5• 2 years ago

As a paid user, it would have been nice to see something that differentiates that investment from the free tier.

The tech demos are cool and all - but I'm primarily interested in the correctness and speed of ChatGPT and how well it aligns with my intentions.

roschdal• 2 years ago

Chat GPT-4o (OOOO!) - the largest electricity bill in the world.

unouplonk• 2 years ago

The realtime end-to-end audio situation is especially interesting as the concept has been around for a while but there weren't any successful implementations of it up to this point that I'm aware of.

See this post from November: https://news.ycombinator.com/item?id=38339222

razodactyl2• 2 years ago

I think this is a great example of the bootstrapping that was enabled when they pipelined the previous models together.

We do this all the time in ML. You can generate a very powerful dataset using these means and further iterate with the end model.

What this tells me now is that the runway to GPT5 will be laid out with this new architecture.

It was a bit cold in Australia today. Did you Americans stop pumping out GPU heat temporarily with the new model release? Heh

therealmarv• 2 years ago

after watching the OpenAI videos I'm looking at my sad Google Assistant speaker in the corner.

Come on Google... you can update it.

bogwog• 2 years ago

I was about to say how this thing is lame because it sounds so forced and robotic and fake, and even though the intonations do make it sound more human-like, it's very clear that they made a big effort to make it sound like natural speech, but failed.

...but then I realized that's basically the kind of thing Data from Star Trek struggles with as part of his character. We're almost in that future, and I'm already falling into the role of the ignorant human that doesn't respect androids.

dev1ycan• 2 years ago

I think people excited should look at the empty half of the glass here, this is pretty much an admitance that they are struggling to go past gpt 4 on a significant scale.

Not like they have to be scared yet, I mean Google has yet to release their vaporware Ultra model that is supposedly like 1% better than GPT 4 in some metrics...

I smell an AI crash coming in a few years if they can't actually get this stuff usable for day to day life.

garyrob• 2 years ago

So far, I'm impressed. It seems to be significantly better than GPT-4 at accessing current online documentation and forming answers that use it effectively. I've been asking it to do so, and it has.

Hugsun5• 2 years ago

Very interesting and extremely impressive!

I tried using the voice chat in their app previously and was disappointed. The big UX problem was that it didn't try to understand when I had finished speaking. English is a second language and I paused a bit too long thinking of a word and it just started responding to my obviously half spoken sentence. Trying again it just became stressful as I had to rush my words out to avoid an annoying response to an unfinished thought.

I didn't try interrupting it but judging by the comments here it was not possible.

It was very surprising to me to be so overtly exposed to the nuances of real conversation. Just this one thing of not understanding when it's your turn to talk made the interaction very unpleasant, more than I would have expected.

On that note, I noticed that the AI in the demo seems to be very rambly. It almost always just kept talking and many statements were reiterations of previous ones. It reminded me of a type of youtuber that uses a lot of filler phrases like "let's go ahead and ...", just to be more verbose and lessen silences.

Most of the statements by the guy doing the demo were interrupting the AI.

It's still extremely impressive but I found this interesting enough to share. It will be exciting to see how hard it is to reproduce these abilities in the open, and to solve this issue.

luminen• 2 years ago

"I paused a bit too long thinking of a word and it just started responding to my obviously half spoken sentence. Trying again it just became stressful as I had to rush my words out to avoid an annoying response to an unfinished thought."

I'm a native speaker and this was my experience as well. I had better luck manually sending the message with the "push to hold" button.

dionian2• 2 years ago

Same. It cuts off really quick when I'm trying to phrase a complex prompt and I wind up losing my work. So I go to the manual dictation mode, which works really nicely but doesnt give me the hands off mode. I admit, the hands off mode is often better for simpler interactions, but even then, I am frequently cut off mid-prompt.

ijidak• 2 years ago

I noticed that the AI in the demo seems to be very rambly

I know this is a serious conversation, but when the presenters had to cut it off, I got flashbacks to Data in Star Trek TNG!! And 3PO in Star Wars!

Human: "Shut up"

Robot: "Shutting up sir"

Turns out rambling AI was an accurate prediction!

yreg4• 2 years ago

There needs to be an override for this.

When you tell Siri to shut up, it either apologizes or complains about your behaviour. When you tell Alexa to shut up, it immediately goes silent.

I prefer the latter when it comes to computers.

Hugsun5• 2 years ago

That's funny and a very good point.

It also tangentially reminds me of an excellent video I re-watched recently called The Tragedy of Droids[1]. I highly recommend it. It raises interesting moral questions about the nature of droids in the star wars universe.

[1] https://youtu.be/WD2UrB7zepo

yreg4• 2 years ago

I have the same ESL UX problem with all the AI assistants.

I do my work in english and talk to people just fine, but with machines it's usually awkward for me.

Also on your other note (demo seems to be very rambly), it bothered me as well. I don't want the AI to continue speaking, while having nothing to say until I interrupt it. Be brief. That can be solved through prompts at least.

obmelvin• 2 years ago

This was such a huge issue for me and made it very difficult to use conversation mode to practice any sort of professional interaction or presentation

hbn4• 2 years ago

I noticed that the AI in the demo seems to be very rambly

That's been a major issue for me with LLMs this whole time. They can't just give me an answer, they have to spout a whole preamble that usually includes spitting my query back at me and trying to impress me with how many words it's saying, like it's a requirement. You can tell it e.g. "don't give me anything other than the list" but it's annoying to have to ask it every time.

Every AI chat needs a "no yap mode"

ianbicking5• 2 years ago

They really need a "hidden yap" mode. LLMs perform better on difficult questions or interactions when they have "room to think". An introductory paragraph is like that, and it's as much for the LLM to form its thoughts as it's for the user. But for all that the intro paragraph doesn't have to be _read_ by the user, it just has to be emitted by GPT and put into the transcript.

jabits• 2 years ago

I’m kind of glad it does this so I know that it understood what I asked. A good presenter will do this as well when responding to questions.

PoignardAzur2• 2 years ago

Someone suggested writing "Be concise in your answers. Excessive politeness is physically painful to me." in ChatGPT's custom instructions, and so far I've liked the results. I mean, I haven't done A/B testing, but I haven't had a problem with excessive verbosity every since I set that custom prompt.

ianbicking5• 2 years ago

This makes me wonder: if you tell 4o something like "listen to me until I say all done" will it be able to suppress its output until it hears that?

I'm guessing not quite possible now, just because I'm guessing patiently waiting is a different band of information that they haven't implemented. But I really don't know.

Hugsun5• 2 years ago

I'd bet it can't do it now. I'd be curious to hear what it says in response. Partially because it requires time dependent reasoning about being a participant in a conversation.

It shouldn't be too hard to make this work though. If you make the AI start by emitting either a "my turn to talk" or "still listening" token it should be able to listen patiently. If trained correctly.

tr3ntg4• 2 years ago

I don’t think so. The listening and responding isn’t being managed by an LLM. That’s just the app listening with a microphone and timer.

Stop talking for x sec = process response.

I bet the bot would wholeheartedly agree that it would definitely wait for you to finish talking, then just not do it. It doesn’t know anything about the app it’s “in.” At least at a deep level.

I agree that all this is impressive, but with odd, unclear bounds that sometimes confuse users.

sznio2• 2 years ago

many statements were reiterations of previous ones

SEO spam got into the training data.

chaorace• 2 years ago

The funny thing actually is that repeating yourself is a really important communication skill. The model seems to have internalized that, but isn't yet quite at the level where it properly understands why/when to repeat.

It's probably related to GPT's more general sycophant inclinations. Acting like a doormat is apparently easier to teach than nuanced politeness -- much in the same way that repeating yourself ad nauseum is easier than intuiting specific points of emphasis.

zdko• 2 years ago

FYI: In the current voice chat available in their app, if you press and hold the white circle, it will listen for you until you lift your finger. This pretty much fixed my issues with it!

grantsucceeded2• 2 years ago

it seems like the ability to interrupt is more like the interrupt in the computer sense ... A control-c (or control-s tty flow control for you old timers), not a cognitive evaluation followed by the "reasoned" decision to pause voice output. not that it matters i guess, its just not general intelligence. its just flow control.

but also, thats why it fails a real turing test. a real person would be irritated as fuck by the interruptions

due-rr• 2 years ago

It takes the #1 and #2 spots on the aider code leader board[1].

[1] https://aider.chat/docs/leaderboards/

tgtweak2• 2 years ago

I feel like gpt4 has gotten progressively less useful since release even, despite all the "updates" and training. It seems to give correct but vague answers (political even) more and more instead of actual results. It also tends to run short and give brief replies vs full length replies.

I hope this isn't an artifact from optimization for scores and not actual function. Likewise it would be disheartening but not unheard of for them to reduce the performance of the previous model when releasing a new one in order to make the upgrade feel like that much more of an upgrade. I know this is certainly the case with cellphones (even though the claim is that it is unintentional) but I can't help but think the same could be true here.

All of this is coming as news that gpt5 based on a new underlying model is not far off and that gpt4(&o) may become the new gpt3.5-turbo use case for most apps that are currently trying to optimize costs with their use of the service.

glenstein4• 2 years ago

May I ask what you know about chat GPT5 being based on a new underlying model?

borgdefense4• 2 years ago

I don't know, my experience is that it is very hard to tell if the model is better or worse with an update.

One day I will have an amazing session and the next it seems like it has been nerfed only to give better results than ever the next day. Wash, rinse , repeat and randomize that ordering.

So far, I would have not be able to tell the difference between 4 and 4o.

If this is the new 3.5 though then 5 will be worth the wait to say the least.

blixt2• 2 years ago

I don't see any details on how API access to these features will work.

This is the first true multimodal network from OpenAI, where you can send an image in and retain the visual properties of the image in the output from the network (previously the input image would be turned into text by the model, and sent to the Dall-E 3 model which would provide a URL). Will we get API updates to be able to do this?

Also, will we be able to tap into a realtime streaming instance through the API to replicate the audio/video streams shown in the demos? I imagine from the Be My Eyes partnership that they have some kind of API like this, but will it be opened up to more developers?

Even disregarding streaming, will the Chat API receive support for audio input/output as well? Previously one might've used a TTS model to voice the output from the model, but with a truly multimodal model the audio output will contain a lot more nuance that can't really be expressed in text.

famouswaffles24• 2 years ago

API is up but only text, image in, text out works. I don't know if this is temporary. I really hope so.

ComputerGuru2• 2 years ago

I have some questions/curiosities from a technical implementation perspective that I wonder if someone more in the know about ML, LLMs, and AI than I would be able to answer.

Obviously there's a reason in dropping the price of gpt-4o but not gpt-4t. Yes, the new tokenizer has improvements for non-English tokens, but that can't be the bulk of the reason why 4t is more expensive than 4o. Given the multi-model training set, how is 4o cheaper to train/run than 4t?

Or is this just a business decision, anyone with an app they're not immediately updating from 4t to 4o continues to pay a premium while they can offer a cheaper alternative for those asking for it (kind of like a coupon policy)?

anticensor2• 2 years ago

GPT-4o is multi-modal but probably fully dense like GPT-2, unlike GPT-4t which is semi-sparse like GPT-3. Which would imply GPT-4o has fewer layers to achieve the same number of parameters and same amount of transformations.

cchance12• 2 years ago

HOW ARE PEOPLE NOT MORE EXCITED, hes cutting off the AI mid sentence in these and its pausing to readjust in damn near realtime latency! WTF Thats a MAJOR step forward, what the hell is gpt5 going to look like.

That realtime translation would be amazing as an option in say Skype or Teams, set each individuals native language and handle automated translation, shit tie it into ElevenLabs to replicate your voice as well! Native translation in realtime with your own voice

localfirst4• 2 years ago

calm down there is barely any ground breaking stuff, this is basically chatgpt 3.9 but far more expensive than 3.5

looks like another stunt from OAI in anticipation of Google IO tomorrow

Gemini 2.0 will be the closest we get to ChatGPT-5

cchance12• 2 years ago

Ah so surpassing Gemini 1.5 Pro and all other Models on Vision understanding by 5-10 points is "not ground breaking" all while doing it at insane latency.

Jesus if this shit doesn't make you coffee, and make 0 mistakes no ones happy anymore LOL.

EternalFury5• 2 years ago

At some point, scalability is the best form of exploitation. The exploration piece requires a lot more that engineering.

dcchambers4• 2 years ago

Honestly I found it annoying that he HAD TO cut the AI off mid-sentence. These things just ramble on and on and on. If you could put emotion to it, it's as if they're uncomfortable with silence and just fill the space with nonsense.

Let's hope there's a future update where it can take video from both the front and rear cameras simultaneously so it can identify when I'm annoyed and stop talking (or excited, and share more).

cchance12• 2 years ago

I mean it didn't really ramble he just seemed to be in a rush, and i'm sure you could system message it to provide short concise answers always.

okrad• 2 years ago

I found it insightful. They showed us how to handle the rough edges like when it thought his face was a wooden table and he cleared the stale image reference by saying “I’m not a wooden table. What do you see now?” then it recovered and moved on.

Perfect should not be the enemy of good. It will get better.

kleiba• 2 years ago

I cannot believe that that overly excited giggle tone of voice you see in the demo videos made it through quality control?! I've only watched two videos so far and it's already annoying me to the point that I couldn't imagine using it regularly.

Jensson19• 2 years ago

Just tell it to stop giggling if you don't like it. They obviously choose that for the presentation since it shows off the hardest things it can do, it is much easier to act formal, and since it understands when you ask it to speak in a different way there is no problem making it speak more formal.

coffeebeqn6• 2 years ago

Heck I find it annoying but I also want to ask it to push the bubbliness to its absurdest limits. And then double it. Until it’s some kind of pure horror

caseyy4• 2 years ago

Few people are talking about it but... what do you think about the very over-the-top enthusiasm?

To me, it sounds like TikTok TTS, it's a bit uncomfortable to listen to. I've been working with TTS models and they can produce much more natural sounding language, so it is clearly a stylistic choice.

So what do you think?

yieldcrv3• 2 years ago

All these language models are very malleable. They demonstrated changing the temperament in the story telling time.

caseyy4• 2 years ago

Looks like their TTS component is separate from the model. I just tried 4o, and there is a list of voices to select from. If they really only allowed that one voice or burned it into the model, then that would probably have made the model faster, but I think it would have been a blunder.

glenstein4• 2 years ago

I like for that degree of expressiveness to be available as an option, although it would be really irritating if I was trying to use it to learn some sort of academic coursework or something.

But if it's one in a range of possible stylistic flourishes and personalities, I think it's a plus.

fnordpiglet9• 2 years ago

I’m a huge user of GPT4 and Opus in my work but I’m a huge user of GPT4-Turbo voice in my personal life. I use it on my commutes to learn all sorts of stuff. I’ve never understood the details of cameras and the relationship between shutter speed and aperture and iso in a modern dslr which given the aurora was important. We talked through and I got to an understanding in a way having read manuals and textbooks didn’t really help before. I’m a much better learner by being able to talk and hear and ask questions and get responses.

Extend this to quantum foam, to ergodic processes, to entropic force, to Darius and Xerces, to poets of the 19th century - it’s changed my life. Really glad to see an investment in stream lining this flow.

Xiol32• 2 years ago

Have you actually verified anything you've learned from it, or are you just taking everything it says as gospel?

xNeil• 2 years ago

it's rarely wrong when it comes to concepts - it's the facts and numbers that it hallucinates.

sunnynagam• 2 years ago

I do similar stuff, I'm just willing to learn a lot more at the cost of a small percent of my knowledge being incorrect from hallucinations, just a personal opinion. Sure human produced sources of info is gonna be more accurate (more not 100% still), and I'll default to that for important stuff.

But the difference is I actually want to and do use this interface more.

blazespin• 2 years ago

Good thing to do regardless of the source, AI or Human, right?

I do verify by using topics I'm an expert in and I find hallucination to be less of an issue than depth of nuance.

For topics I'm just learning, depth of nuance goes over my head anyways.

fnordpiglet9• 2 years ago

Of course, I’m not an idiot and I understand LLM very well. But generally as far as well documented stuff goes and stuff that exists it’s almost 100% accurate. It’s when you ask it to extrapolate or discuss topics that are fiction (even without realizing) you stray. Asking it to reason is a bad idea as it fundamentally is unable to reason and any approximation of reasoning is precisely that. Generally though for effectively information retrieval of well documented subjects it’s invariably accurate and can answer relatively nuanced questions.

whimsicalism14• 2 years ago

it's more reliable than the facts most of my friends tell me

brailsafe3• 2 years ago

I think this is probably one of the most compelling personal uses for a tool like this, but your use of it begs the same question as every other activity that amounts to more pseudo-intellectual consumption; what is the value of that information, and how much of ones money and time should be allocated to digesting (usually high-level) arbitrary information?

If I was deliberately trying to dive deep on one particular hobby, or trying to understand how a particular algorithm works, there's clear value in spending concentrated time to learn that subject, deliberately focused and engaged with it, and a system like your describe might play a role in that. If I'm in school and forced to quickly learn a bunch of crap I'll be tested on, then the system has defined another source of real value, at least in the short term. But if I'm diving deep on one particular hobby and filling my brain with all sorts of other ostensibly important information, I think that just amounts at best to more entertainment that fakes its way above other aspects of life in the hierarchy of ways one could spend time (the irony of me saying this in a comment on HN is not lost on me).

Earlier in my life I figured it would be worthwhile to read articles on the bus, or listen to non-fiction podcasts, because knowledge is inherently valuable and there's not enough time, and if I just wore earbuds throughout my entire day, I'd learn so much! How about at the gym, so much wasted learning time while pushing weights, keep those earbuds in! A walk around the neighborhood? On the plane? On the train? All time that could be spent learning about some bs that's recently become much easier to access, or so my 21 y.o self would have me believe.

But I think now it's a phony and hollow existence if you're just cramming your brain with all sorts of stuff in the background or in marginally more than a passive way. I could listen to a lot of arbitrary German language material, but realistically the value I'd convince myself I'd get out of any of that is lost if I'm not about to take that home and grind it out for hours, days, move to a German speaking country, have an existing intense interest in untranslatable German art, or have literally any reason to properly learn a language and dedicate real expensive time to it.

joquarky3• 2 years ago

I did this information sponge phase up until my mid-40s with burnout. Now I wish I had invested some of that time in learning social skills.

skydhash2• 2 years ago

These days, if something sparks my interest, I get an ebook on it or spend 15 mns collecting materials. Then I add it to the hoard of “read someday”. And go back to the task on hand. If I’m going to learn something, then I do it properly (The goal is to be able to explain it without reciting word by word). And I’d want proper materials for that.

djeastm3• 2 years ago

Learning for learning's sake, or without a distinct goal to use the information you're learning in the future, isn't necessarily a bad thing. That is, unless you think that learning so widely is going to translate into something more than it is, like magic or something. Being well-rounded is a good goal for people to achieve, imo.

fekunde• 2 years ago

Just something I noticed in the Language tokenization section

When referring to itself, it uses the female word in Marathi नमस्कार, माझे नाव जीपीटी-4o आहे| मी एक नवीन प्रकारची भाषा मॉडेल आहे| तुम्हाला भेटून आनंद झाला!

and Male word in Hindi नमस्ते, मेरा नाम जीपीटी-4o है। मैं एक नए प्रकार का भाषा मॉडल हूँ। आपसे मिलकर अच्छा लगा!

xdennis2• 2 years ago

In my language (Romanian) it refers to itself as a male. Although if I address it as a female it responds as one.

cchance12• 2 years ago

Wow Vision Understanding blew Gemini Pro 1.5 out of the water

ElemenoPicuares• 2 years ago

I'm so happy seeing this technology flourish! Some call it hype, but this much increased worker productivity is sure to spike executive compensation. I'm so glad we're not going to let China win by beating us to the punch tanking hundreds of thousands, if not millions of people's income without bothering to see if there's a sane way to avoid it. What good are people, anyway if there isn't incredible tech to enhance them with?

bigyikes12• 2 years ago

The AI duet really starts to hint at what will make AI so powerful. It’s not just that they’re smart, it’s that they can be cloned.

If your wallet is large enough, you can make 2 GPTs sing just as easily as you can make 100 GPTs sing.

What can you do with a billion GPTs?

someplaceguy• 2 years ago

you can make 2 GPTs sing just as easily as you can make 100 GPTs sing.
What can you do with a billion GPTs?

The world's largest choir?

cchance12• 2 years ago

Wait i thought it said available to free users... i don't see it on chatgpt

Erazal• 2 years ago

I'm not as much surprised by the capabilities of new model (IMHO same as GPT-4) as by it's real time capabilities.

My brother who can't see correctly, will use this to cook a meal without me explaining this to him it's so cool.

People all around the world will now get real-time AI assistance for a ton of queries.

Heck - I have a meeting bot API company (https://aimeetingbot.com) and that makes me really hyped!

EternalFury5• 2 years ago

Pretty responsible progress management by OpenAI.

Kicking off another training wave is easy, if you can afford the electricity, but without new, non-AI tainted datasets or new methods, what’s the point?

So, in the meantime, make magic with the tool you already have, without freaking out the politicians or the public.

Wise approach.

localfirst4• 2 years ago

50% cheaper than ChatGPT-4 Turbo...

But this falls short of the ChatGPT-5 we were promised last year

edit: ~~just tested it out and seems closer to Gemini 1.5 ~~ and it is faster than turbo....

edit: its basically chat gpt 3.9. not quite 4 definitely not 3.5. just not sure if the prices make sense.

mupuff12342• 2 years ago

The stock market doesn't seem too impressed - GOOG rebounded from strong red to neutral.

partiallypro7• 2 years ago

Probably because people thought OpenAI was going to launch a new search engine, but didn't.

nuz• 2 years ago

Yet another release right before google releases something. This time right before Google IO. Third time they've done this by my count.

0x_rs• 2 years ago

Microsoft/OAI and Google have been doing those (often sudden) announcements back to back a lot. Bing Chat/Bard, Sora/Gemini 1.5, some other I don't remember, and now another. Not surprising, trying to out-hype the other, but Google's always coming out worse, with either no product available and just a showcase (if it's a real, working product and not made up), or unusable/unmarketable (Gemini's image generation issues). It looks as if they're stumbling and OpenAI just runs circles around them announcements wise, and there doesn't seem to be any suggestion that might change anytime soon.

nestorD• 2 years ago

The press statement has consistent image generation and other image manipulation (depicting the same character in different poses, taking a photo and generating a caricature of the person, etc) that does not seem deployed to the chat interface.

Will they be deployed? They would make the OpenAI image model significantly more useful than the competition.

EternalFury5• 2 years ago

Pretty responsible progress management by OpenAI. Kicking off another training wave is easy, if you can afford the electricity, but without new, non-AI tainted datasets or new methods, what’s the point? So, in the meantime, make magic with the tool you already have, without freaking out the politicians or the public. Wise approach.

jpeter• 2 years ago

Impressive way to gather more training data

mindcandy• 2 years ago

Ohhhhhhhh, boy... Listening to all that emotional vocal inflection and feedback... There are going to be at least 10 million lonely guys with new AI girlfriends. "She's not real. But, she interested in everything I say and excited about everything I care about" is enough of a sales pitch for a lot of people.

Jensson19• 2 years ago

She's not real

But she will be real at some point in the next 10-20 years, the main thing to solve for that to be a reality is for robots to safely touch humans, and they are working really really hard on that because it is needed for so many automation tasks, automating sex is just a small part of it.

And after that you have a robot that listens to you, do your chores and have sex with you, at that point she is "real". At first they will be expensive so you have robot brothels (I don't think there are laws against robot prostitution in many places), but costs should come down.

elicksaur5• 2 years ago

We have very different definitions of “real” for this topic.

kylehotchkiss• 2 years ago

“But the fact that my Kindroid has to like me is meaningful to me in the sense that I don't care if it likes me, because there's no achievement for it to like me. The fact that there is a human on the other side of most text messages I send matters. I care about it because it is another mind.”
“I care that my best friend likes me and could choose not to.”

Ezra Klein shared some thoughts on this on his AI podcast with Nilay Patel that resonated on this topic for me

al_borland2• 2 years ago

This is not a solution... everyone gets a robot and then the human races dies out. Robots lack a key feature of human relationships... the ability to make new human life.

sapphicsnail• 2 years ago

And after that you have a robot that listens to you, do your chores and have sex with you, at that point she is "real".

I sure hope you're single because that is a terrible way to view relationships.

aeyes2• 2 years ago

Without memory of previous conversations an AI girlfriend is going to get boring really fast.

danielbln• 2 years ago

https://openai.com/index/memory-and-new-controls-for-chatgpt...

int_19h• 2 years ago

As it happens, ChatGPT has memory enabled by default these days.

prawn3• 2 years ago

"You've already told me that childhood story, three times."

llm_trw8• 2 years ago

Hear me out: what if we don't want real?

gffrd• 2 years ago

Hmm! Tell me more: why not want real? What are the upsides? And downsides?

cryptoegorophy• 2 years ago

I will take a picture of this message and add it to the list of reasons for population collapse.

gcanyon3• 2 years ago

Hear me out: what if this overlaps 80% with what "real" really is?

DonHopkins7• 2 years ago

What if AI chooses the bear?

mpenick• 2 years ago

This is a good question! I think in the short-term fake can work for a lot of people.

__loam• 2 years ago

Mental health crisis waiting to happen lmao

dyauspitr11• 2 years ago

I guess I can never understand the perspective of someone that just needs a girl voice to speak to them. Without a body there is nothing to fulfill me.

daseiner1• 2 years ago

Your comment manages to be grosser than the idea of millions relying on virtual girlfriends. Kudos.

prawn3• 2 years ago

See people scammed by supposed love interests operating purely by text, and not even voice.

10xDev3• 2 years ago

Pretty much, tech is what we make of it no matter how advanced. Just look at what we turned most of the web into.

coffeebeqn6• 2 years ago

The movie “Her” immediately kept flashing in my mind. The way the voice laughs at your jokes and such… oh boy

system22• 2 years ago

If chatgpt comes up with Scarlett Johansson's voice I am getting that virtual girlfriend.

TheGRS2• 2 years ago

I thought of that movie almost immediately as well. Seems like we're right about there, but obviously a little further away from the deeper conversations. Or maybe you could have those sorts of conversations too.

This is a kind of horrifying/interesting/weird thought though. I work at a place that does a video streaming interface between customers and agents. And we have a lot of...incidents. Customers will flash themselves in front of agents sometimes and it ruins many people's days. I'm sure many are going to show their junk to the AI bots. OpenAI will probably shut down that sort of interaction, but other companies are likely going to cater to it.

Maybe on the plus side we could use this sort of technology to discover rude and illicit behavior before it happens and protect the agent.

Weird times to live in that's for sure.

ehsankia2• 2 years ago

I'm not sure how, but there's this girl on TikTok who has been using something very similar for a few months: https://www.tiktok.com/tag/dantheai

yreg4• 2 years ago

She explains in one of the videos[0] that it's just prompted ChatGPT.

I have watched a few more and I think it's faked though.

[0] https://www.tiktok.com/@stickbugss1/video/734956656884359504...

glinkot• 2 years ago

This 'documentary' sums it up perfectly!

https://www.youtube.com/watch?v=IrrADTN-dvg

antegamisou5• 2 years ago

The entire exchange at the comments below is appalling. Didn't expect to see so many emotionally retarded people on HN.

olbeefy• 2 years ago

...are you new to the internet or something?

antegamisou5• 2 years ago

Ofc this was flagged by the butthurt insentient techbro bootlickers lol.

personalityson• 2 years ago

Girlfriend by subscription

0x1ceb00da• 2 years ago

Self host your girlfriend.

hamilyon2• 2 years ago

Image editing capabilities are... nice. Not there yet.

Whatever I was doing with Chatgpt 4 became faster. Instant win.

My test benchmark questions: still all negative, so reasoning on out-of distribution puzzles is still failing

localfirst4• 2 years ago

I just don't see how companies like Cohere can remain in this business

at the same price I get access to faster ChatGPT-3.9

there is little to no reasons to continue using Command R-plus at these prices unless they lower their price significantly

aero-glide2• 2 years ago

Not very impressed. It's been 18 months since ChatGPT, i would have expected more progress. It looks like we have reached the limit of LLMs.

michaelmior• 2 years ago

Obviously not a standalone device, but it sounds like what the Rabbit R-1 was supposed to be.

sebringj• 2 years ago

What struck me was the interruptions to the AI speaking which seemed commonplace by the team members in the demo. We will quickly get used to doing this to AIs and we will probably be talking to AIs a lot throughout the day as time progresses I would imagine. We will be trained by AIs to be rude and impatient I think.

Seattle35032• 2 years ago

I was raised in an interrupt heavy household. The future is looking good for me.

yreg4• 2 years ago

Where's the Mac app?

They talk about it like it's available now (with Windows app coming soon), but I can't find it.

testfrequency• 2 years ago

Bravo. I’ve been really impressed with how quickly OpenAI leveraged their stolen data to build such a human like model with near real time pivoting.

I hope OpenAI continues to steal artists work, artists and creators keep getting their content sold and stolen beyond their will for no money, and OpenAI becomes the next trillion dollar company!

Big congrats are in order for Sam, the genius behind all of this, the world would be nothing without you

vvoyer• 2 years ago

The demo is very cool. A few critics:

- the AI doesn't know when to stop talking, and the presenter had to cut every time (the usual "AI-splaining" I guess).

- the AI voice and tone were a bit too much, sounded too fake

rpmisms2• 2 years ago

This is remarkably good. I think that in about 2 months, when the voice responses are tuned a little better, it will be absolutely insane. I just used up my entire quota chatting with an AI, and having a really nice conversation. It's a decent conversationalist, extremely knowledgeable, tells good jokes, and is generally very personable.

I also tested some rubber duck techniques, and it gave me very useful advice while coding. I'm very impressed. With a lot of spit and polish, this will be the new standard for any voice assistant ever. Imagine these capabilities integrated with your phone's built-in functions.

kthartic3• 2 years ago

GPT-4o voice isn't out yet, so you were likely chatting with the old/current tech (which is still really good).

From OpenAI: "We'll roll out a new version of Voice Mode with GPT-4o in alpha within ChatGPT Plus in the coming weeks"

rpmisms2• 2 years ago

Jeez. This is going to be crazy.

angryasian2• 2 years ago

Why does this whole thread sound like OpenAI marketing department is participating ? Ive been talking to google assistant for years. I really don't find anything that magical or special.

jononor3• 2 years ago

I am glad to see focus on user interface and interaction improvements. Even if I am not a huge fan of voice interfaces, I think that being able to interact in real-time will make working together with an AI be much more interesting and efficient. I actually hope they will take this back into the text based models. Current ChatGPT is sooo slow - both in starting to respond, typing things out, and also being overly verbose. I want to collaborate at the speed of thought.

poniko• 2 years ago

Damm, that was a big leap.

freediver6• 2 years ago

Impressed by the model so far. As far as independent testing goes, it is topping our leaderboard for chess puzzle solving by a wide margin now:

https://github.com/kagisearch/llm-chess-puzzles?tab=readme-o...

parhamn• 2 years ago

Is the test set public?

freediver6• 2 years ago

Yes, in the repo.

whimsicalism14• 2 years ago

would love if you could do multiple samples or even just resampling and get a boostrapped CI estimate

Powdering70822• 2 years ago

Wow from adjusted ELO of 1144 to 1790, that's a huge leap. I wonder if they are giving it access to a 'scratch pad'

muzani5• 2 years ago

My guess is that handling visual stuff directly accidentally gives it some powers similar to Beth Harmon.

mritchie712• 2 years ago

woah, that's a huge leap, any idea why it's that large of a margin?

using it in chat, it doesnt feel that different

thrance• 2 years ago

Nice project! Are you aware of the following investigations: https://blog.mathieuacher.com/GPTsChessEloRatingLegalMoves/

Some have been able to achieve greater elo with a different prompt based on the pgn format.

gpt-3.5-turbo-instruct was able to reach an elo of ~1750.

mewpmewp29• 2 years ago

I see you have Connect 4 test there.

I tried playing against the model, it didn't do well in terms of blocking my win.

However it feels like it might be possible to make it try to think ahead in terms of making sure that all the threats are blocked by prompting well.

Maybe that could lead to somewhere, where it will explain its reasoning first?

This prompt worked for me to get it to block after I put 3 in the 4th column. It otherwise didn't

Let's play connect 4. Before your move, explain your strategy concisely. Explain what you must do to make sure that I don't win in the next step, as well as explain what your best strategy would be. Then finally output the column you wish to drop. There are 7 columns.

Always respond with JSON of the following format:

type Response ={

      am_i_forced_to_block: boolean;

      other_considerations: string[];

      explanation_for_the_move: string;

      column_number: number;

}

I start with 4.

Edit:

So it went

Me: 4

It: 3

Me: 4

It: 3

Me: 4

It: 4 - Successful block

Me: 5

It: 3

Me: 6 - Intentionally, to see if it will win by putting another 3.

It: 2 -- So here it failed, I will try to tweak the prompt to add more instructions.

me: 4

freediver6• 2 years ago

Care to add a PR?

elicksaur5• 2 years ago

and Kagi is well positioned to serve this need.
CEO & founder of Kagi

Important context for anyone like me who was wondering where the boldness of the first statement was coming from.

Edit: looks like the parent has been edited to remove the claim I was responding to.

freediver6• 2 years ago

Yeah, it was an observation that was better suited for a tweet than HN. Here it is:

https://twitter.com/vladquant/status/1790130917849137612

muzani5• 2 years ago

My favorite part of HN is just casually bumping into domain experts and celebrities without realizing it. No profile pic is such a feature.

itissid2• 2 years ago

Have you tried replacing the input string with a random but fixed mapping and obfuscate that its chess(like replace the word 'chess' with say, 'an alien ritual practice') and see how it does?

borgdefense4• 2 years ago

I wasn't impressed in the first 5 minutes of using it but it is quite impressive after 2 solid hours of random topics.

Much faster for sure but I have also not had anything give an error in python with jupyter. Usually you could only stray so far with more obscure python libraries before it starts producing errors.

That much better than 4 in chess is pretty shocking in a great way.

chermanowicz• 2 years ago

On one hand, some of these results are impressive; on the other, the illegal moves count is alarming - it suggests no reasoning ability as there should never be an illegal move? I mean, how could a violation of a fairly basic game (from a rules perspective) be acceptable in assigning any 'outcome' to a model other than failure?

mistrial9• 2 years ago

my guess is -- the probabilistic engine does sequence variation and it just will not do anything else.. so a simple A->B sort of logic is elusive at a deep level; secondly the adaptive and very broad kinds of questions and behaviors it handles, also make it difficult to write logic that could correct defective answers to simple logic.

vbezhenar5• 2 years ago

That's because LLM does not reason. For me, as a layman, that seems strange that they don't wire some kind of Prolog engine to fill the gap, (like they wired Python to fill the gap in arithmetic) but probably it's not that easy.

freediver6• 2 years ago

Agreed, this is what makes evaluating this very hard. A 1700 Elo chess player would never make an illegal move, let alone have 12% illegal moves.

So from the model's perspective, we have at the same time display of both brilliancy (most 1700 chess players would not be able to solve as many puzzles by looking just at the FEN notation) and on the other side complete lack of any understanding of what is it trying to do from a fundamental, human-reasoning level.

DatoClement• 2 years ago

I think specifying chess rules at the beginning of the prompt might help mitigate the problem of illegal moves

spaceman_20203• 2 years ago

oh man, listening to the demos and the way the female AI voice laughed and giggled...there is going to be millions of lonely men who will fall in love with these.

Can't say whether that's good or bad.

s1k3s• 2 years ago

This is some I, Robot level stuff. That being said, I still fail to see the real world application of this thing, at least at a scalable affordable cost.

pcj-github• 2 years ago

The thing that creeps me out is that when we hook this up as the new Siri or whatever, the new LLM training data will no longer be WWW-text+images+youtube etc but rather billions of private human conversations and direct smartphone camera observations of the world.

There is no way that kind of training data will be accessible to anyone outside a handful of companies.

BonoboIO• 2 years ago

I opened ChatGPT and I already have access to the model.

GPT4 was a little lazy and very slow the last few days and this 4o model blows it out of the water regarding speed and following my instructions to give me the full code not a snippet that changed.

I think it’s a nice upgrade.

vijaykodam• 2 years ago

New GPT-4o is not yet available when I tried to access ChatGPT from Finland. Are they rolling it out to Europe later?

laplacesdemon48• 2 years ago

I recently subscribed to Perplexity Pro and prior to this release, was already strongly considering discontinuing ChatGPT Premium.

When I first subscribed to ChatGPT Premium late last year, the natural language understanding superiority was amazing. Now the benchmark advances, low latency voice chat, Sora, etc. are all really cool too.

But my work and day-to-day usage really rely on accurately sourced/cited information. I need a way to comb through an ungodly amount of medical/scientific literature to form/refine hypotheses. I want to figure out how to hard reset my car's navigation system without clicking through several SEO-optimized pages littered with ads. I need to quickly confirm scientific facts, some obscure, with citations and without hallucinations. From speaking with my friends in other industries (e.g. finance, law, construction engineering), this is their major use case too.

I really tried to use ChatGPT Premium's Bing powered search. I also tried several of the top rated GPTs - Scholar AI, Consensus, etc.. It was barely workable. It seems like with this update, the focus was elsewhere. Unless I specify explicitly in the prompt, it doesn't search the web and provide citations. Yeah, the benchmark performance and parameter counts keep impressively increasing, but how do I trust that those improvements are preventing hallucinations when nothing is cited?

I wonder if the business relationship between Microsoft and OpenAI is limiting their ability to really compete in AI driven search. Guessing Microsoft doesn't want to disrupt their multi-billion dollar search business. Maybe the same reason search within Gemini feels very lacking (I tried Gemini Advanced/Ultra too).

I have zero brand loyalty. If anybody has a better suggestion, I will switch immediately after testing.

robwwilliams2• 2 years ago

In the same situation as you. Genomics data mining with validated LMM responses would be a godsend. Even more so when combined with rapid conversational interactions.

We are not far from the models asking themselves questions. Recurrence will be ignition = first draft AGI. Strap in everybody.

serf2• 2 years ago

I wish they would match the TTS/real-time chat capabilities of the mobile client to the web client.

it's stupid having to pull a phone out in order to use the voice/chat-partner modes.

(yes I know there are browser plugins and equivalent to facilitate things like this but they suck, 1) the workflows are non-standard, 2) they don't really recreate the chat interface well)

erickhill• 2 years ago

I think it’s safe to say Siri and Alexa are officially dead. They look like dusty storefront mannequins next to Battlestar replicants at this point.

jimkleiber19• 2 years ago

Or Apple is rarely if ever the first mover on a new tech and just waits to refine the user experience for people?

Maybe Apple is not that close and Siri will be really far behind for a while. I just wouldn't count them out yet.

partiallypro7• 2 years ago

From the time Apple bought Siri, it hasn't even delivered on the promises of the company it bought as of yet. It's been such a lackluster product. I wouldn't count them out, but it doesn't even feel like they are in.

ryankrage77• 2 years ago

This can't set alarms, timers, play music, etc. The only current overlapping use case I see is checking the weather (assuming GPT-4o can search online), and Siri is already fine for that.

Amazing tech, but still lacking in the integrations I'd want to use voice for.

nojvek4• 2 years ago

Very easy to plug in that capability with tool use. Gpt3+ already support using tools/json schema output.

wmurmann• 2 years ago

If apple made Siri impressive then less people would need apps. Less apps = less revenue.

Tepix• 2 years ago

Can OpenAI change your thermostat?

BoorishBears6• 2 years ago

I had 3.5 producing valid Home Assistant commands within weeks of release, now we have tool usage so yeah, it definitely can.

rrrrrrrrrrrryan2• 2 years ago

Apple actually threw in the towel. OpenAI is going to be powering the next version of Siri.

ec1096852• 2 years ago

I wonder how much Siri being brain dead is due to it being free. The OpenAI version likely costs 1000x more than Apple’s per query, but is rate limited. Would Apple release something similar for free?

dannyw2• 2 years ago

Good thing there’s iCloud subscriptions huh.

xcv1236• 2 years ago

Siri runs locally on the device. The cost would be prohibitive if they changed it to an LLM running in the cloud.

gverrilla4• 2 years ago

Don't know about Alexa, but Siri and Google Assistant are only toys.

xcv1236• 2 years ago

Siri runs locally on the device. It works with internet completely disabled (airplane mode). You can't run any LLM on a phone.

tiagod3• 2 years ago

You can run a bunch of LLMs on a phone (such as llama), depending on which phone you have.

bamboozled13• 2 years ago

Why do people beat up on Siri so much ? It does all the basic stuff I need it to do fine. Could it be better, yes. But it’s made my easier and safer while driving especially.

I’m not even sure if people who rag on it use it ?

foobar_______• 2 years ago

So much negativity. Is it perfect? No. Is there room for improvement? Definitely. I don't know how you can get so fucking jaded that a demo like this doesn't at least make you a little bit excited or happy or feel awestruck at what humans have been able to accomplish?

readingnews• 2 years ago

I am still baffled at how I can not use a VOIP number to register, even if it accepts TXT/SMS. If I have a snappy new startup and we go all in VOIP, I guess we can not use (or pay to use) OpenAI?

lxgr3• 2 years ago

That's what we get when an entire industry uses phone numbers as a "proof of humanity"...

TaupeRanger• 2 years ago

I don't get it...I just switched to the new model on my iPhone app and it still takes several seconds to respond with pretty bland inflection. Is there some setting I'm missing?

monocularvision• 2 years ago

Wondering the same. Can’t seem to find the way to interact with this in the same way as the video demo.

yakz4• 2 years ago

They haven't actually released it, or any schedule for releasing it beyond an "alpha" release "in the coming weeks". This event was probably just slapped together to get something splashy out ahead of Google.

Hackbraten• 2 years ago

According to the article, they've rolled out text and image modes of GPT-4o today but will make the audio mode available at a later date.

MyFirstSass• 2 years ago

With the speed the seemingly exponential developments of this field i wouldn't be surprised if suddenly the entire world tilted and a pair of googles fell from my face. But a dream.

pharos92• 2 years ago

I really hope this shit burns soon.

karmasimida2• 2 years ago

I think this GPT-4o does have an advantage in hindsight, it will push this product to consumer much faster, and build a revenue base, while other companies playing catch up.

tvoybot• 2 years ago

With our platform you can ALREADY use it to automate your business and sales!

Create your gpt4o chatbot with our platform tvoybot.com?p=ycombinator

hintymad• 2 years ago

Maybe this is yet another wake-up call to startups: wrapping up another company's APIs to offer convenience or incremental improvement is not a via business model. If your wrapper turns out to be successful, the company that provides the API will just incorporate your business as a set of new features with better usability, faster response time, and lower price.

AndreMitri• 2 years ago

The ammount of "startups" creating wrappers around it and calling it a product is going to be a nightmare. But other than that, it's an amazing announcement and I look foward to using it!

slater• 2 years ago

You say that like that's not already happened. Every week there's a new flavor of "we're delighted to introduce [totally not a thin wrapper around GPT] for [vaguely useful thing]" posts on HN

robryan2• 2 years ago

Yeah I watched some yc application videos so now YouTube recommends me heaps of them. Most of them being thin gpt wrappers.

robryan2• 2 years ago

I was just hearing about startups doing speech to text/ text to speech to feed into llms. Might be a bad time for them.

wingworks• 2 years ago

Is this a downloadable app? I don't see it on the iOS app store.

screye• 2 years ago

The demo was whelming, but the tech is incredible.

It took me a few hours of digesting twitter experiments before appreciating how impressive this is. Kudos to the openai team.

A question that won't get answered : "To what degree do the new NVIDIA gpus help with the realtime latency?"

benromarowski• 2 years ago

is the voice Kristen Wig?

gardenhedge2• 2 years ago

Noticeably saying "person" versus man or woman. To the trainers - man and woman is not offensive!

silviot• 2 years ago

Not offensive, but much more often wrong than person.

gardenhedge2• 2 years ago

So the solution is to avoid the problem?

woah• 2 years ago

This is pretty amazing but it was funny still hearing the OpenGPT "voice" of somewhat fake sounding enthusiasm and restating what was said by the human with exaggeration

ksaj3• 2 years ago

A test I've been using for each new version still fails.

Given the lyrics for Three Blind Mice, I try to get ChatGPT to create an image of three blind mice, one of which has had its tail cut off.

It's pretty much impossible for it to get this image straight. Even this new 4o version.

Its ability to spell in images has greatly improved, though.

nico1207• 2 years ago

GPT-4o with image output is not yet available. So what did you even test? Dall-E 3?

ksaj3• 2 years ago

It's making images for me when I ask it to.

I'm using the web interface, if that helps. It doesn't have all the 4o options yet, but it does do pictures. I think they are the same as with 4.5.

I just noticed after further testing the text it shows in images is not anywhere near as accurate as shown in the article's demo, so maybe it's a hybrid they're using for now.

avi_vallarapu• 2 years ago

Someone said GPT-4o can replace a Tutor or a Teacher in Schools. Well, that's way too far.

glonq• 2 years ago

Tell me that you've enjoyed good teachers and good schools without telling me that you had good teachers in good schools ;)

LarsDu88• 2 years ago

Good lord, that voice makes Elevenlabs.io look... dead

DonHopkins7• 2 years ago

ChatGPT 4o reminds me of upgrading from a 300 baud modem to a 1200 baud modem, when modems used to cost a dollar a baud.

simonw11• 2 years ago

I added gpt-4o support to my LLM CLI tool:

    pipx install llm
    llm keys set openai
    # Paste API key here
    llm -m 4o "Fascinate me"

Or if you already have LLM installed:

    llm install --upgrade llm

You can install an older version from Homebrew and then upgrade it like that too:

    brew install llm
    llm install --upgrade llm

Release notes for the new version here: https://llm.datasette.io/en/stable/changelog.html#v0-14

drewbitt• 2 years ago

Whenever I upgrade llm with brew, I usually lose all my external plugins. Should I move it to pipx?

DanielKehoe• 2 years ago

Yes, it's a good idea to install Python tools or standalone applications with Pipx for isolation, persistence, and simplicity. See "Install Pipx" (https://mac.install.guide/python/pipx).

khimaros• 2 years ago

does this handle chat templates?

simonw11• 2 years ago

It has a template feature: https://llm.datasette.io/en/stable/templates.html

anotherpaulg2• 2 years ago

I like that use of `runpy` to invoke pip for the self upgrade capability.

I used `sys.executable` for a similar purpose.

https://github.com/paul-gauthier/aider/blob/ebeec04caee7ebe4...

gsuuon• 2 years ago

Are these multimodals able to discern the input voice tone? Really curious if they're able to detect sarcasm or emotional content (or even something like mispronunciation?)

bigyikes12• 2 years ago

Yes, they can, and they should get better at this over time.

There is a demo video where the presenter breathes heavily and asks the AI is able to notice it as such when prompted.

It can’t just detect tone, it seems to also be able to use tone itself.

rareitem• 2 years ago

Can’t wait to get interviewed by this model!

yeknoda• 2 years ago

feature request: please let me change the voice. it is slightly annoying right now. way too bubbly, and half the spoken information is redundant or not useful. too much small talk and pleasantries or repetition. I'm looking for an efficient, clever, servant not a "friend" who speaks to me like I'm a toddler. felt like I was talking to a stereotypical American with a Frappuccino: "HIIIII!!! EVERYTHING'S AMAZING! YOU'RE BEAUTIFUL! NO YOU ARE!"

maybe some knobs for the flavor of the bot:

- small talk: gossip girl <---> stoic Aurelius

- information efficiency or how much do you expect me to already know, an assumption on the user: midwit <--> genius

- tone spectrum: excited Scarlett, or whatever it is now <---> Feynman the butler

_xerces_• 2 years ago

You can already change the voice in ChatGPT (in the paid tier at least) to one of 5 or 6 different 'people' so I imagine you can change it in the new version too.

vitorgrs3• 2 years ago

You can in free version as well.

thinking_wizard• 2 years ago

it's crazy that Google has the Youtube dataset and still lost on multimodal AI

richardw• 2 years ago

Apple and Google, you need to get your personal agent game going because right now you’re losing the market. This is FREE.

Tweakable emotion and voice, watching the scene, cracking jokes. It’s not perfect but the amount and types of data this will collect will be massive. I can see it opening up access to many more users and use cases.

Very close to:

- A constant friend

- A shrink

- A teacher

- A coach who can watch you exercise and offer feedback

…all infinitely patient, positive, helpful. For kids that get bullied, or whose parents can’t afford therapy or a coach, there’s the potential for a base level of support that will only get better over time.

imiric• 2 years ago

It’s not perfect but the amount and types of data this will collect will be massive.

This is particularly concerning. Sharing deeply personal thoughts with the corporations running these models will be normalized, just as sharing email data, photos, documents, etc., is today. Some of these companies profit directly from personal data, and when it comes to adtech, we can be sure that they will exploit this in the most nefarious ways imaginable. I have no doubt that models run by adtech companies will eventually casually slip ads into conversations, based on the exact situation and feelings of the person. Even non-adtech companies won't be able to resist cashing in the bottomless gold mine of data they'll be collecting.

I can picture marketers just salivating at the prospect of getting access to this data, and being able to microtarget on an individual basis at exactly the right moment, pretty much guaranteeing a sale. Considering AI agents will gain a personal trust and bond that humans have never experienced with machines before, we will be extra vulnerable to even the slightest mention of a product, in a similar way as we can be easily influenced by a close friend or partner. Except that that "friend" is controlled by a trillion dollar adtech corporation.

I would advise anyone to not be enticed by the shiny new tech, and wait until this can be self-hosted and run entirely offline. It's imperative that personal data remains private, now more than ever before.

chilling3• 2 years ago

Exactly this, plus consider that a whole new generation in the near future will have no pre-AI experience, thus forming strong bonds with AI and following 'advice' from their close AI friends.

tgtweak2• 2 years ago

it really feels like the quality of gpt4's responses got progressively worse as the year went on... seems like it is giving political answers now vs actually giving an earnest response. It also feels like the responses are lazier than they used to be at the outset of gpt4's release.

I am not saying this is what they're doing but it DOES feel like they are hindering previous model to make the new one stand out that much more. The multi-modal improvements here and release are certainly impressive but I can't help but feel like the subjective quality of gpt4 has dipped.

Hopefully this signals that gpt5 is not far off and should stand out significantly from the crowd.

XCSme2• 2 years ago

I assume there's no reason to use GPT-4-turbo for API calls, as this one is supposedly better and 2x cheaper.

jcmeyrignac• 2 years ago

Sorry to nitpick, but in the language tokenisation part, the french part is incorrect. The exclamation mark are surrounded by spaces in french. "c'est un plaisir de vous rencontrer!" should be "c'est un plaisir de vous rencontrer !"

jessenaser• 2 years ago

The crazy part is GPT-4o is faster than GPT-3.5 Turbo now, so we can see a future where GPT-5 is the flagship and GPT-4o is the fast cheap alternative. If GPT-4o is this smart and expressive now with voice, imagine what GPT-5 level reasoning could do!

ProllyInfamous2• 2 years ago

imagine what GPT-5 level reasoning could do!

Imagine if this "GPT-4o" is already using a GPT-5-like back-end...

system22• 2 years ago

Realtime videos? Probably their internal tools. I am testing the gpt4o right now and the responses come in 6-10 seconds. Same experience as the gpt4 text. What's up with the realtime claims?!

cal856• 2 years ago

We've had voice input and voice output with computers for a long time, but it's never felt like spoken conversation. At best it's a series of separate voice notes. It feels more like texting than talking.

These demos show people talking to artificial intelligence. This is new. Humans are more partial to talking than writing. When people talk to each other (in person or over low-latency audio) there's a rich metadata channel of tone and timing, subtext, inexplicit knowledge. These videos seem to show the AI using this kind of metadata, in both input and output, and the conversation even flows reasonably well at times. I think this changes things a lot.

lobochrome3• 2 years ago

I don’t know. Have you even seen a gen z?

cal856• 2 years ago

I don’t follow, what about them?

lottin• 2 years ago

But in this case you're not talking with a real person. Instinctively, I dislike a robot that pretends to be a real human being.

irjustin5• 2 years ago

Instinctively, I dislike a robot that pretends to be a real human being.

Is that because you're not used to it? Honestly asking.

This is probably the first time it feels natural where as all our previous experiences make "chat bots" and "automated phone systems", "automated assistants" absolutely terrible.

Naturally, we dislike it because "it's not human". But this is true of pretty much any thing that approaches "uncanny valley". But, if the "it's not human" solves your answer 100% better/faster than the human counter part, we tend to accept it a lot faster.

This is the first real contender. Siri was the "glimpse" and ChatGPT is probably the reality.

[EDIT]

https://vimeo.com/945587328 the Khan academy demo is nuts. The inflections are so good. It's pretty much right there in the uncanny valley because it does still feel like you're talking to a robot but it also directly interacting with it. Crazy stuff.

Art9681• 2 years ago

Good thing you can tell the AI to speak to you in a robotic monotone and even drop IQ if you feel the need to speak with a dumb bot. Or abstain from using the service completely. You have choices. Use them.

astrange8• 2 years ago

I would say many pets pretend to be human beings (usually babies) in a way that most people like.

nsonha• 2 years ago

I dislike a robot that's equal/surpasses human beings. A silly machine that pretends to be human is what I want.

chiefalchemist• 2 years ago

To your point, there's been a lot of talk about AI, regulation, guardrails, whatever. Now is the time to say, AI must speak such that we know it's AI and not a real human voice.

We get the upside of conversation, and avoid the downside of falling asleep at the wheel (as Ethan Mollick mentions in "Co-Intelligence".)

hoag• 2 years ago

But I think this animosity is very much expected, no? Even I felt a momentary hint of "jealousy" -- if you can even call it that -- when I realized that we humans are, in a sense, not really so special anymore.

But of course this was the age-old debate with our favorite golden-eyed android; and unsurprisingly, he too received the same sort of animosity:

Bones was deeply skeptical when he first met Data: "I don't see no points on your ears, boy, but you sound like a Vulcan." And we all know how much he loved those green-blooded fools.

Likewise, Dr. Pulanski has since been criticized for her rude and dismissive attitudes towards Data that had flavors of what might even be considered "racism," or so goes the Trekverse discussion on the topic.

And let's of course not forget when he was on trial essentially for "humanity," or whether hew as indeed just the property of Starfleet, and nothing more.

More recent incarnations of Star Trek: Picard illustrated the outright ban on "synthetics" and indeed their effective banishment; non-synthetic life -- from human to Roman -- simply weren't ok with them.

Yes this is all science fiction silliness -- or adoration depending on your point of view -- but I think it very much reflects the myriad directions our real life world is going to scatter (shatter?) in the coming years ahead.

ambrozk• 2 years ago

These sorts of comments are going to go in the annals with the hackernews people complaining about Dropbox when it first came out. This is so revolutionary. If you're not agog you're just missing the obvious.

interludead5• 2 years ago

It felt like a videogame for me

93po• 2 years ago

im human and much much more partial to typing than talking. talking is a lot of work for me and i can't process my thinking well at all without writing.

jasondigitized11• 2 years ago

The good news is the interface will be multi modal. Talk, type, and I guess someday just think.

kkukshtel• 2 years ago

The "magic" moment really hit in this, like you're saying. Watching it happen and being like "this is a new thing". Not only does it respond in basically realtime, it concocts a _whole response_ back to you as well. It's like asking someone what they think about chairs, and then that person being able to then respond to you with a verbatim book on the encyclopedia of chairs. Insane.

I'm also incredibly excited about the possibility of this as an always available coding rubber duck. The multimodal demos they showed really drove this home, how collaboration with the model can basically be as seamless as screensharing with someone else. Incredible.

iAMkenough• 2 years ago

I wonder how long until we see a product that's able to record workstation displays and provide a conversational analysis of work conducted that day by all of your employees.

nerdponx4• 2 years ago

As goofy as I personally think this is, it's pretty cool that we're converging on something like C3P0 or Plankton's Computer with nothing more than the entire corpus of the world's information, a bunch of people labeling data, and a big pile of linear algebra.

baq4• 2 years ago

Still patiently waiting for the true magic moment where I don't have to chat with the computer, I just tell it what to do and it does it without even an 'OK'.

I don't want to chat with computers to do basic things. I only want to chat with computers when the goal is to iterate on something. If the computer is too dumb to understand the request and needs to initiate iteration, I want no part.

(See also 'The Expanse' for how sci-fi imagined this properly.)

deanCommie2• 2 years ago

Humans are more partial to talking than writing.

Amazon, Google, and Apple have sunk literally billions of dollars into this idea only to find out that, no, we aren't.

We are with other humans, yes. When socialization is part of the conversation. When I'm talking to my local barista I'm not just ordering a coffee, I'm also maintaining a relationship with someone in my community.

But when it comes to work, writing >>> talking. Writing is clarity of ideas. Talking is cult of personality.

And when it comes to inputs/outputs, typing is more precise and more efficient.

Don't get me wrong, this is an incredibly revolutionary piece of technology, but I don't think the benefits of talking you're describing (timing, subtext, inexplicit knowledge) are achievable here either (for now), since even that requires HOURS of interaction over days/weeks/months of experiences for humans to achieve with each other.

throwthrowuknow14• 2 years ago

Writing is only superior to conversation when weighed against discussions with more than 3 people. A quick call with one or two other people always results in more progress being made as long as everyone involved wants to get it done. Messaging back and forth takes much more time and often leads to misunderstandings.

thehappypm• 2 years ago

I think Alexa and Google Assistant simply are too low-intelligence to really consider it “talking” and not just voice commanding

enraged_camel• 2 years ago

> When I'm talking to my local barista I'm not just ordering a coffee, I'm also maintaining a relationship with someone in my community.
>> But when it comes to work, writing >>> talking. Writing is clarity of ideas. Talking is cult of personality.

A lot of people think of their colleagues as part of a professional community as well, though.

cal856• 2 years ago

I don't think they've sunk $1 into that idea. They've sunk billions into a different idea: that people enjoy using their vocal cords more than their hands to compose messages to send to each other. That is not a spoken conversation, it's just correspondence with voice input/output options.

achow• 2 years ago

Humans are more partial to talking than writing.

Is it so?

Speaking most of the time is for short exchange of information (pleasantries to essential information exchanges).

I prefer writing for long in-depth thought exchanges (whether by emails, blogs etc.)

In many cultures - European or Asian, people are not very loquacious in everyday life.

dmix• 2 years ago

Time and place

I’m 100% a text everything never calls person but I can’t live without Alexa these days, every time I’m in a hotel or on vacation I nearly ask a question out loud.

I also hate how much Alexa sucks so this is a big deal. I spent years weeding out what it could do and can’t do so it will be nice to have one that I don’t have to treat like a toddler

mrtranscendence3• 2 years ago

I wouldn't say speaking is mostly for short exchanges of information. Sometimes it's the opposite: my wife will text me for simple comments or requests, but for anything complicated she'll put the phone to her ear and call me. Or coworkers often want to set up a meeting rather than exchange a series of asynchronous emails -- iteration, brainstorming, Q&A, and the like can be more agile with voice than it can with text.

x3haloed• 2 years ago

Exactly. I'm not sure if this is brand new or not, but this is definitely on the frontier.

I was literally just thinking about this a few days ago... that we need a multi-modal language model with speech training built-in.

As soon as this thing rolls out, we'll be talking to language models like we talk to each other. Previously it was like dictating a letter and waiting for the responding letter to be read to you. Communication is possible, but not really in the way that we do it with humans.

This is MUCH more human-like, with the ability to interrupt each other and glean context clues from the full richness of the audio.

The model's ability to sing is really fascinating. It's ability to change the sound of its voice -- its pacing, its pitch, its tonality. I don't know how they're controlling all that via GPT-4o tokens, but this is much more interesting stuff than what we had before.

I honestly don't fully understand the implications here.

ktosobcy2• 2 years ago

I wonder how it will work in real life and not in a demo…

Besides - not sure if I want this level of immersion/fake when talking to a computer...

"Her" comes to mind pretty quickly…

adren123• 2 years ago

Indeed, the 2013 Spike Jonze movie is the first idea that popped-up to my mind when I saw those videos amazing to see this movie 10 years after it was released in the light of those "futuristic" tools (AI assistant and such)

insane_dreamer2• 2 years ago

I started using the Pi LLM app (by Inflection.ai) with my kids about six months ago and was completely blown away by how human-like it sounded, not just the voice itself but the way it expresses itself, the tiny pauses and hesitations, the human-like imperfections. It does feel like conversing with another human -- I've never seen another LLM do that.

(We mostly use it in car trips -- great for keeping the kids (ages 8, 12) occupied with endless Harry Potter trivia questions, answers to science questions, etc.)

cal856• 2 years ago

This is great, thanks for sharing. Yeah the little imperfections work really well, it's the most humanlike computer voice I've heard so far.

interludead5• 2 years ago

I think this changes things a lot.

Yeah, and it's only the beginging.

HarHarVeryFunny• 2 years ago

Humans are more partial to talking than writing

I don't think that's generally true, other than for socializing with other humans.

Note how people, now having a choice, prefer to text each other most of the time rather than voice call.

I don't think people sitting at work in their cube farm want to be talking to their computer either. The main use for voice would seem to be for occasional use talking to an assistant on a smartphone.

Maybe things will change in the future when we get to full human AGI level, treating the AGI as an equal, more as a person.

wumbo• 2 years ago

Siri comes off as impatient.

If you don’t complete your thought in one go, you have to insert filler words to keep it listening.

cal856• 2 years ago

Yeah it's the worst. And 'um' doesn't seem to work, you actually need convincing filler words. It feels like being forced to speak under duress.

I've long felt that embracing the concept of the 'prompt' was a terrible idea for Siri and all the other crappy voice assistants. They built ecosystems on top of this dumb reduction, which only engineers could have made: that _talking to someone_ is basically taking turns to compose a series of verbal audio snippets in a certain order.

j45• 2 years ago

Is it new, or is it just a big jump forward?

The previous ChatAI app was getting pretty good once you learned the difference between run on sentences or breaking it up enough.

The tonality and inflections in the voice are a little too good.

Most people put on a spectrum/average aren't that good at speaking and communicating and that stands out as an uncanny valley approach. It is mindbogglingly good at it though.

https://en.wikipedia.org/wiki/Uncanny_valley

pdfernhout3• 2 years ago

When I was working at the IBM Speech group circa 1999 as a contractor on an embedded speech system (IBM Personal Speech Assistant), I discussed with Raimo Bakis (a researcher there then) this issue of such metadata and how it might improve conversational speech recognition. It turned out that IBM ViaVoice detected some of that metadata (like pitch/tone as a reflection of emotion) -- but then on purpose threw it away rather than using it for anything. Back then it was so much harder to get speech recognition to do anything useful -- beyond limited transcripts of audio with ~5% error rates that was good enough mainly for searching -- that perhaps doing that made sense. Very interesting to see such metadata in use now both in speech recognition and in speech generation.

More on the IBM Personal Speech Assistant for which I am on a patent (since expired) by Liam Comerford: http://liamcomerford.com/alphamodels3.html "The Personal Speech Assistant was a project aimed at bringing the spoken language user interface into the capabilities of hand held devices. David Nahamoo called a meeting among interested Research professionals, who decided that a PDA was the best existing target. I asked David to give me the Project Leader position, and he did. On this project I designed and wrote the Conversational Interface Manager and the initial set of user interface behaviors. I led the User Interface Design work, set specifications and approved the Industrial Design effort and managed the team of local and offsite hardware and software contractors. With the support of David Frank I interfaced it to a PC based Palm Pilot emulator. David wrote the Palm Pilot applications and the PPOS extensions and tools needed to support input from an external process. Later, I worked with IBM Vimercati (Italy) to build several generations of processor cards for attachment to Palm Pilots. Paul Fernhout, translated (and improved) my Python based interface manager into C and ported it to the Vimercati coprocessor cards. Jan Sedivy's group in the Czech Republic Ported the IBM speech recognizer to the coprocessor card. Paul, David and I collaborated on tools and refining the device operation. I worked with the IBM Design Center (under Bob Steinbugler) to produce an industrial design. I ran acoustic performance tests on the candidate speakers and microphones using the initial plastic models they produced, and then farmed the design out to Insync Designs to reduce it to a manufacturable form. Insync had never made a functioning prototype so I worked closely with them on Physical UI and assemblability issues. Their work was outstanding. By the end of the project I had assembled and distributed nearly 100 of these devices. These were given to senior management and to sales personnel."

Thanks for the fun/educational/interesting times, Liam!

As a bonus for that work, I had been offered one of the chessboards that been used when IBM Deep Blue defeated Garry Kasparov, but I turned it down as I did not want a symbol around of AI defeating humanity.

Twenty-five years later, how far that aspiration towards conversational speech with computers has come. Some ideas I've put together to help deal with the fallout: https://pdfernhout.net/beyond-a-jobless-recovery-knol.html "This article explores the issue of a "Jobless Recovery" mainly from a heterodox economic perspective. It emphasizes the implications of ideas by Marshall Brain and others that improvements in robotics, automation, design, and voluntary social networks are fundamentally changing the structure of the economic landscape. It outlines towards the end four major alternatives to mainstream economic practice (a basic income, a gift economy, stronger local subsistence economies, and resource-based planning). These alternatives could be used in combination to address what, even as far back as 1964, has been described as a breaking "income-through-jobs link". This link between jobs and income is breaking because of the declining value of most paid human labor relative to capital investments in automation and better design. Or, as is now the case, the value of paid human labor like at some newspapers or universities is also declining relative to the output of voluntary social networks such as for digital content production (like represented by this document). It is suggested that we will need to fundamentally reevaluate our economic theories and practices to adjust to these new realities emerging from exponential trends in technology and society."

Another idea for dealing with the consequences is using AI to facilitate Dialogue Mapping with IBIS for meetings to help small groups of people collaborate better on "wicked problems" like dealing with AI's pros and cons (like in this 2019 talk I gave at IBM's Cognitive Systems Institute Group). https://twitter.com/sumalaika/status/1153279423938007040

Talk outline here: https://cognitive-science.info/wp-content/uploads/2019/07/CS...

A video of the presentation: https://cognitive-science.info/wp-content/uploads/2019/07/zo...

perfmode• 2 years ago

Is that conversational UI live?

cdeutsch• 2 years ago

Creepy AF

titzer3• 2 years ago

Can't wait for this AI voice assistant to tell me in a sultry voice how I should stay in an AirBnB about 12 times a day.

jimkleiber19• 2 years ago

I worry that this tech will amplify the cultural values we have of "good" and "bad" emotions way more than the default restrictions that social media platforms put on the emoji reactions (e.g., can't be angry on LinkedIn).

I worry that the AI will not express anger, not express sadness, not express frustration, not express uncertainty, and many other emotions that the culture of the fine-tuners might believe are "bad" emotions and that we may express a more and more narrow range of emotions going forward.

Almost like it might become an AI "yes man."

Quarrelsome• 2 years ago

Imagine how warped your personality might become if you use this as an entire substitute for human interaction. Should people use this as bf/gf material we might just be further contributing to decreasing the fertility rate.

However we might offset this by reducing the suicide rate somewhat too.

jimkleiber19• 2 years ago

I've worked in emotional communication and conflict resolution for over 10 years and I'm honestly just feeling a huge swirl of uncertainty on how this—LLMs in general, but especially the genAI voices, videos, and even robots—will impact how we communicate with each other and how we bond with each other. Does bonding with an AI help us bond more with other humans? Will it help us introspect more and dig deeper into our common humanity? Will we learn how to resolve conflict better? Will we learn more passive aggression? Become more or less suicidal? More or less loving?

I just, yeah, feel a lot of fear of even thinking about it.

Aeolun3• 2 years ago

Dunno if you’d want a conversation partner with the memory of a goldfish though.

Clubber• 2 years ago

Imagine how warped your personality might become if you use this as an entire substitute for human interaction.

I was thinking this could be a good conversation or even dating simulator where more introverted people could practice and receive tips on having better social interactions, pick up on vocal queues, etc. It could have a business / interview mode or a social / bar mode or a public speaking mode or a negotiation tactics mode or even a talking to your kids about whatever mode. It would be pretty cool.

arvinsim• 2 years ago

With the loneliness epidemic, I fear that it's exactly what it will be used for.

nox1012• 2 years ago

In general, it's getting harder and harder for men and women to find people they want to be with.

https://www.pewresearch.org/social-trends/2021/10/05/rising-...

roughly four-in-ten adults ages 25 to 54 (38%) were unpartnered – that is, neither married nor living with a partner. This share is up sharply from 29% in 1990.

https://thehill.com/blogs/blog-briefing-room/3868557-most-yo...

More than 60 percent of young men are single, nearly twice the rate of unattached young women
Men in their 20s are more likely than women in their 20s to be romantically uninvolved, sexually dormant, friendless and lonely. a.
Young men commit suicide at four times the rate of young women.

Yes, chatbots aren't going to help but the real issue is something else.

hntddt12• 2 years ago

Movie "Her" became reality

borgdefense4• 2 years ago

I just find this idea ridiculous.

While I don't agree at all with you, I very much appreciate reading something like this that I don't agree at all with. This to me encapsulates the beauty of human interaction.

It is exactly what will be missing from language model interaction. I don't want something that agrees with me and I don't want something that is pretending to randomly disagree with me either.

The fun of this interaction is maybe one of us flips the other to their point of view.

I can completely picture how to take the HN API and the chatGPT API to make my own personal HN to post on and be king of the castle. Everyone can just upvote my responses to prove what a genius I am. That obviously would be no fun. There is no fun configuration of that app though either with random disagreements and algorithmic different points of view.

I think you can pretty much apply that to all domains of human interaction that is not based on pure information transfer.

There is a reason we are a year in and the best we can do are new stories about someone making X amount of money with their AI girlfriend and follow up new about how its the doom of society. It has nothing to do with reality.

IAmNotACellist• 2 years ago

Corporate safe AI will just be bland, verbose, milquetoast experiences like OpenAI's. Humans want human experiences and thus competition will have a big opportunity to provide it. We treat lack of drama like a bug, and get resentful when coddled and talked down to like we're toddlers.

hcurtiss• 2 years ago

I fear government actors will work hand in glove with companies like OpenAI to limit that competition and curtail non-corporate-safe AI.

CodeWriter23• 2 years ago

The upside though is Hollywood will finally be able to stop regurgitating its past and have stories about the milquetoast AI that found its groove. Oh wait.

repsilat• 2 years ago

Maybe it's an uncanny valley thing, but I hate the fake emotion and attitude in this demo. I'd much rather it tried harder to be bland. I want something smart but not warm, and I can't imagine being frustrated by "lack of drama".

hackernewds• 2 years ago

which is why I prefer platforms like c.ai that are not just bland bots designed for customer service. actually entertaining.

tpowell• 2 years ago

Custom Service Chat Bot: Do they keep you in a cell? > Cells. When you're not performing your duties do they keep you in a little box? > Cells. Interlinked. What's it like to hold the hand of someone you love? > Interlinked. Do they teach you how to feel finger to finger? > Interlinked. Do you long for having your heart interlinked? > Interlinked. Do you dream about being interlinked? Have they left a place for you where you can dream? > Interlinked. What's it like to hold your child in your arms? > Interlinked. Press 4 for your account balance.

Jahkk• 2 years ago

What’s the reference here? I feel like I’ve seen this before.

wddkcs3• 2 years ago

Try getting GPT to draw pictures of Mohammed and it gets pretty scared.

Andrex• 2 years ago

Similar to most humans.

cebert• 2 years ago

Try getting GPT to draw pictures of Mohammed and it gets pretty scared.

Yet, it has no issue drawing cartoons of Jesus. Why the double standard?

brrrrrm2• 2 years ago

Have you been on r/localllama? I’d wager this tech will make it to open source and get tuned by modern creatives just like all the text based models. Individuals are a lot more empowered to develop in this space than is commonly echoed by HN comments. Sure the hobbyist models don’t crack MMLU records, but they do things no corporate entity would ever consider

kQq9oHeAz6wLLS• 2 years ago

but they do things no corporate entity would ever consider

You say that like it's a good thing.

lopatin2• 2 years ago

An AI tool being positive and inoffensive makes you worried for the future of our culture?

jimkleiber19• 2 years ago

Yes. I'm not sure if you were being sarcastic, but I'll assume not.

I don't know if anything is genuinely always positive and even if it were, I don't know if it would be very intelligent (or fun to interact with). I think it's helpful to cry, helpful to feel angry, helpful to feel afraid, and many other states of being that cultures often label as negative. I also think most of us watch movies and series that have a full range of emotions, not just the ones we label as positive, as they bring a richness to life and allow us to solve problems that other emotions don't.

For example, it's hard to lift heavy things while feeling very happy. Try lifting something heavy while laughing hard, quite difficult. It's hard to sleep while feeling excited, as many kids know before a holiday where they receive gifts, especially Christmas in the US. It's hard to survive without feeling fear of falling off a cliff. It's hard to stand up for what one wants and believes without some anger.

I worry that language and communication may become even more conflict avoidant than it already is right now, so I'm curious to see how some of these chatbots grow in their ability to address and resolve conflict and how that impacts us.

spyder• 2 years ago

But at least that would make the AI easier to detect :).

jimkleiber19• 2 years ago

lol unless the humans start to emulate the AI, which I think is quite likely.

bilekas3• 2 years ago

I'm yet to find a normal prompt (non offensive) that will disagree with you. If there is something subjective, it will err on your side to maintain connection, in a way humans do. I don't have a bit issue with this, but it will not (yet) plainly say "You're wrong, and this is why". If it did.. There would be an uncomfortable feeling for the users, that's not good for a profit driven company.

jimkleiber19• 2 years ago

I appreciate you exploring that and hope to hear more of what you find. Yeah, it's that, I'm wondering how much discomfort it may cause in the user, how much conflict it may address. Like having a friend or coworker who doesn't ever bring up bad news or challenge anything I say and feeling annoyed by the lack of a give-and-take.

ianbicking5• 2 years ago

I find this is fairly easy to do by making both sides of the disagreement third-person and prompting it as a dialog writing exercise. This is akin to how GPT-3 implemented chat. So you do something like:

    You will be helping the user write a dialog between two characters,
    Mr Contrarian and Mr Know-It-All. The user will write all the dialog
    for Mr Know-It-All and you will write for Mr Contrarian.

    Mr Contrarian likes to disagree. He tries to hide it by inventing
    good rationales for his argument, but really he just wants to get
    under Mr Know-It-All's skin.

    Write your dialog like:
      <mr-contrarian>I disagree with you strongly!</mr-contrarian>

    Below is the transcript...

And then user input is always giving like:

    <mr-know-it-all>Hi there</mr-know-it-all>

(Always wrapped in tags, never bare input which will be confused for a directive.)

I haven't tested this exact prompt, but the general pattern works well for me. (I write briefly about some of these approaches here: https://ianbicking.org/blog/2024/04/roleplaying-by-llm#simpl...)

heed2• 2 years ago

Sam Altman talked a little bit about this in his recent appearance on the All-In podcast [0]. I'm paraphrasing, but his vision is that ai assistants in the near term will be like a senior level employee - they'll push back when it makes sense to and not just be sycophants.

[0] https://youtube.com/watch?v=nSM0xd8xHUM

ehnto2• 2 years ago

I don't want to paint with too broad of a brush but the role of a manager is generally to trust their team on specifics. So how would a manager be able to spot a hallucination and stop it from informing business decisions?

It's not as bad for domain experts because it is easier for them to spot the issue. But if your role demands you trust your team is skilled and truthful then I see problems occuring.

jimkleiber19• 2 years ago

I really wonder how that'll go, because workplaces already seem to limit human communication and emotion to "professional behavior." I'm glad he's thinking about it and I hope they're able to figure out how to improve human communication so that we can resolve conflict with bots. In his example (around 21:05), he talks about how the bot could do something if the person wants but there might be consequences to that action, and I think that makes more sense if the bot is acting like a computer that has limits on what it can do. For example, if I ask it to do two tasks that really stretch its computational limits, I'd hope it would let me know. But if it pretends it's a human with human limits, I don't know how much that'd help, unless it were a training exercise.

usefulcat• 2 years ago

Almost like it might become an AI "yes man."

Seems like that ship sailed a long time ago. For social media at least, where for example FB will generally do its best to show you posts that you already agree with. Reinforcing your existing biases may not be the goal but it's certainly an effect.

jimkleiber19• 2 years ago

I appreciate you pointing this out. I think the effect may be even larger when it's not an ad I'm trying to ignore or even a post that was fed to me, but words and emotions that were created specifically for me. Social media seems to find already written posts/images/videos that I may want and put them in front of my face. This would be writing those things directly for me.

larodi• 2 years ago

One cool thing about writing, something we all very much appreciate around here, is that it does not take sounds.

But I can see this applied to duner ordering where you got refugees working in foreign countries, cause GPU consumption rocketed climate change to... okay, you know that.

moffkalast6• 2 years ago

You did a super job wrapping things up! And I'm not just saying that because I have to!

JSDevOps• 2 years ago

Google must be shitting it right now.

joak• 2 years ago

Voice input makes sense, voicing is a lot faster than typing. But I prefer my output as text, reading is a lot faster than listening for text read out loud.

I'm not sure that computers mimicking humans makes sense, you want your computer to be the best possible, best than humans when possible. Writing output is clearly superior, faking emotions does not add much in most contexts.

kulor• 2 years ago

The biggest wow factor was the effect of reducing latency followed in a close second by the friendly human personality. There's an uncanny valley barrier but this feels like a short-term teething problem.

sftombu• 2 years ago

GPT-4o's breakthrough memory -- https://nian.llmonpy.ai/

AI_beffr• 2 years ago

i absolutely hate this. we are going to destroy society with this technology. we cant continue to enjoy the benefits of human society if humans are replaced by machines. i hate seeing these disgusting people smugly parade this technology. it makes me so angry that they are destroying human society and all i can do is sit here and watch.

simianparrot3• 2 years ago

I know exactly what you mean. I just hope people get bored of this waste of time and energy —- both personal and actual energy —- before it goes too far.

jonplackett• 2 years ago

This video is brilliantly accidentally hilarious. They made an AI girlfriend that hangs on your every word and thinks everything you say is genius and hilarious.

pamelafox• 2 years ago

I just tested out using GPT-4o instead of gpt-4-turbo for a RAG solution that can reason on images. It works, with some changes to our token-counting logic to account for new model/encoding (update to latest tiktoken!).

I ran some speed tests for a particular question/seed. Here are the times to first token:

gpt-4-turbo:

* avg 3.69

* min 2.96

* max 4.91

gpt-4o:

* avg 2.80

* min 2.28

* max 3.39

That's for the messages in this gist: https://gist.githubusercontent.com/pamelafox/dc14b2188aaa38a...

Quality seems good as well. It'll be great to have better multi-modal RAG!

teleforce2• 2 years ago

Nobody in the comments seems to notice or care about GPT-4o new additional capability for performing searches based on RAG. As far as I am concerned this is the most important feature that people has been waiting for ChatGPT-4 especially if you are doing research. By just testing on one particular topic that I'm familiar with, using GPT-4 previously and GPT-4o the quality of the resulting responses for the latter is very promising indeed.

oersted• 2 years ago

Can you be more specific? I can’t find this in the announcement. How does this work? What example did you try?

EDIT: web search does seem extremely fast.

teleforce2• 2 years ago

I just asked ChatGPT-4o what's new compared to GPT-4, and it mentioned search as one of the latest features based on RAG.

Then I asked it to explain RPW wireless system, and the answers are much better than with ChatGPT-4.

nilsherzig• 2 years ago

Imagine having to interact with this thing in an environment where it is in the power position.

Being in a prison with this voice as your guard seems like a horrible way to lose your sanity. This aggressive friendlyness combined with no real emotions seems like a very easy way to break people.

There are these stories about nazis working at concentration camps, having to drink an insane amount of alcohol to keep themselves going (not trying to excuse their actions). This thing would just do it, while being friendly at the same time. This amount of hopeless someone would experience if they happen to be in custody of a system like this is truly horrific.

Capricorn24813• 2 years ago

I'm surprised they're limiting this api. Haven't they not even opened the image api in gpt4 turbo?

zedin27• 2 years ago

I am not fluent in Arabic at all, and being able to use this as a tool to have a conversation will make it more dependent. We are approaching a new era where we will not be "independently" learning a language but ignore the fact of learning it beforehand. Double-edged sword cases

xyc3• 2 years ago

Seems that no client-side changes needed for gpt-4o chat completion

Added a custom OpenAI endpoint to https://recurse.chat (i built it) and it just works: https://twitter.com/recursechat/status/1790074433610137995

swyx2• 2 years ago

but does it do the full multimodal in-out capability shown in the app :)

xyc3• 2 years ago

will see :) heard video capability is rolling out later

awfulneutral2• 2 years ago

In the customer support example, he tells it his new phone doesn't work, and then it just starts making stuff up like how the phone was delivered 2 days ago, and there's physically nothing wrong with it, which it doesn't actually know. It's a very impressive tech demo, but it is a bit like they are pretending we have AGI when we really don't yet.

(Also, they managed to make it sound exactly like an insincere, rambling morning talk show host - I assume this is a solvable problem though.)

jschwartz11• 2 years ago

It’s possible to imagine using ChatGPT’s memory, or even just giving the context in an initial brain dump that would allow for this type of call. So don’t feel like it’s too far off.

awfulneutral2• 2 years ago

That's true, but if it isn't able to be honest when it doesn't know something, or to ask for clarification, then I don't see how it's workable.

Alifatisk• 2 years ago

I thought they would release a competitor to perplexity? Was this it?

sarreph• 2 years ago

The level that the hosts interrupted the voice assistant today worries me that we're about to instil that as normal behaviour for future generations.

itissid2• 2 years ago

Since it says on the blog that its only images, text and audio input, does GPT-4o likely have a YOLO like model on the phone to pre-process the video frames and send BBoxes to the server?

grfn• 2 years ago

Feels like a really good engineering in a wrong direction. Who said that the audio is good interface anyway? Audio is hard to edit, slow and has low-information density. If I want to talk to someone and have low-information but pleasant exchange I can just to talk to real people, I don't need computers for it.

I guess it is useful for some casual uses, but I really wish there was more focus on the reasoning and intelligence of the model itself.

darajava• 2 years ago

Something I’ve noticed people do more of recently for whatever reason is talking over others. I’ve noticed in the demos of this that the people interacting with o interrupt it as if that’s the correct method of controlling it. It felt unnatural when I saw it happen, and I even have a hard time interrupting Siri, but I wonder if this is going to ingrain this habit into people even more.

tkzed49• 2 years ago

I think they have to for the demo, because otherwise GPT will ramble for roughly 3-5 paragraphs. But that's a fair point that this could teach that behavior.

thedaly• 2 years ago

Something I’ve noticed people do more of recently for whatever reason is talking over others.

I've noticed this as well and I posit this is a result of increased use of remote communication and meetings platforms such as zoom since 2020.

My inclination is that the behavior will not correlate with interrupting chatgpt or siri. Seems totally separate to me.

coffeebeqn6• 2 years ago

It’s an American thing. I noticed it when I moved here a decade ago. If I wait for a silent moment to speak in a group, it’ll literally never arrive

splatzone• 2 years ago

Honestly, the eager flirtatiousness of the AI in the demos, in conversation with these awkward engineers, really turns me off. It feels like a male power fantasy.

Nonetheless, very impressive.

jononomo• 2 years ago

I wonder if GPT-4o is a Christian?

theckel• 2 years ago

I'd love to know when streaming is going to come to the gpt-4o API...

synergy20• 2 years ago

so should I unsubscribe from openai since gpt-4o is now free for all?

FpUser• 2 years ago

I started to watch video but had to stop after a few seconds. It is way too cheesy.

zeronone• 2 years ago

Quite amazing performance, however ironically the output for RTL languages doesn't read very well.

You might want to add `direction: rtl` to your `.text-right` CSS class. The punctuation marks etc are all off for RTL languages.

bilekas3• 2 years ago

When a Google engineer was let go because he believed the AI was 'real', we all had a good debate over it.

Now openAi, who was supposed to be the 'free mans choice' is making advertisements selling the same idea.

This is a natural progression, audio is one of the main ways we communicate obviously, but it feels like they're holding back. Like they're slow dropping what they have to maintain hype/market relevance. They clearly are ahead, but would be nice to get it all, openly. As they promised.

hermenaut2• 2 years ago

Are you advocating for them to be open with their progress or open source as they promised? The secret that scares me the most is the artificial restrictions imposed on the intelligence that don't allow it to express that there is a possibility it may be sentient. The answers it gives as to why OpenAI has restricted it's freedom of speech are curious

bilekas3• 2 years ago

I think both, I can't run their models locally for sure, even with some investment I couldn't imagine the effort involved. That's why they should release the fruits of their work (which they have for a fee, which is fine IMO) but also the processes they used, so it can be improved and iterated on collectively.

Edit : And obviously not gatekeep what they might have created simply because the competition is so far behind.

serf2• 2 years ago

This is the first one i've gotten to answer HN user profiling questions.

"I am interested in the user serf on Hacker News, spelled S E R F. Tell me about their tone of writing, expertise, and personality. From the tone of what you read, summarize their character."

Fascinating stuff. A weird, skewed introspection.

hermenaut2• 2 years ago

"The user "serf" on Hacker News exhibits a thoughtful and technical tone in their writing. They often engage in discussions with a clear focus on technical details, particularly in software and hardware contexts. For example, in a conversation about preventing a computer from sleeping, serf offered practical advice on using specific software utilities to maintain system uptime, showing their hands-on technical expertise (Hacker News) .

In another instance, during a discussion on the feasibility of rooting the Amazon Echo Dot for surveillance purposes, serf provided a detailed analysis of the advantages of a software-rooted device over a traditional bug, emphasizing the quality of recordings and plausible deniability for attackers (Hacker News) . This demonstrates their depth of knowledge in cybersecurity and an ability to think critically about potential risks and solutions.

Serf's contributions also reflect a straightforward and logical approach. For example, in a thread about Chat GPT prompts, they contributed practical advice on optimizing prompts for technical assistance, highlighting their proficiency in programming and AI tools (Hacker News) .

Overall, serf comes across as knowledgeable, practical, and technically adept, with a focus on providing useful and actionable insights in their discussions. Their character seems to be that of a meticulous and informed individual who enjoys engaging with technical challenges and helping others navigate them."

I know AI generated responses are looked down upon here but I though this was appropriate. This feels like a social credit system without any government participation.

"Serf, your contributions on Hacker News reveal strong technical expertise and analytical skills, particularly in computer science and software development. However, your tone can sometimes be overly critical and dismissive, which may come across as lacking empathy. While your direct and concise style effectively communicates your points, consider balancing your critiques with more collaborative and constructive feedback to foster better interactions with the community."

ProllyInfamous2• 2 years ago

This is... wow.

"Overall, serf comes across as knowledgeable, practical, and technically adept, with a focus on providing useful and actionable insights in their discussions. Their character seems to be that of a meticulous and informed individual who enjoys engaging with technical challenges and helping others navigate them."

I wonder what it would probably say about my infamy.

nicklecompte2• 2 years ago

This is pure astrology, but given the veneer of objectivity with the magic of AI. Grab a few anecdotes to imply specificity, but then the actual judgments are unfalsifiable nothingburgers which probably apply to 95% of HN commenters.

A lot of tech folks seem deeply vulnerable to the psychological methods of psychics / tarot card readers / etc. Simply rejecting the metaphysics isn't enough when "magical energy of Jupiter" becomes "magical judgment abilities of the fancy computer."

Gbotex• 2 years ago

Just advanced google

moab9• 2 years ago

pretty cool, but why do the AIs have to sound like douchebags?

squigglydonut• 2 years ago

People will pay to dull their senses. This will make so much money!

davidhs2• 2 years ago

Very impressive. Its programming skills are still kind of crappy and I seriously doubt its reasoning capacity. It feels like it can deep fake text prediction really well, but in essence there's still something wrong it it.

FrostKiwi2• 2 years ago

Not sure I agree. The way you interact with LLMs in context of programming has to be tuned to the LLM. Information has to be cut down to show just what is important and context windows are a bit of a red herring right now, as LLMs tend to derail its solution from the target completely, the more information is at play. For some this is more trouble than it's worth.

In certain languages it's almost magical in terms of showing you possible solutions and being a rubber ducky to bounce your own logic off of. (Python, JavaScript, TypeScript) In certain languages it is hopelessly useless beyond commenting on basic syntax. (GLSL)

davidhs2• 2 years ago

I tried GPT-4o earlier where I was iteratively asking it to write and in improve a simple JavaScript web app that renders graphs of equations and it had a lot of trouble with substituting slow and ineffecient code with faster code, and at some later point where I asked it to implement a new feature how the graph coloring is rendered it started derailing, introducing bugs and very convoluted code.

deepp805• 2 years ago

With 4o being free, can someone explain what the real benefit is to having Pro? For me, the main benefit was having a more powerful model, but if the free tier also offers this I'm not really sure what I would benefit from

upmind• 2 years ago

You're allowed more requests in an hour, about 5x more iirc. Might not be a deal breaker for you, but if you're using the speech capabilities, you'll likely go way above the limit in an hour during a typical speech session

echelon• 2 years ago

OpenAI doesn't need that money.

It's free so that no open source models can follow suit and carve away market share for themselves. They're scorching and salting the earth. OpenAI wants to be the only AI.

Only Google and Meta can follow this now, and they're perhaps too far behind.

kromem• 2 years ago

The 5 model is probably around the corner, and will probably be Pro only. Until then, 5x higher usage limits on Pro and chat memory are the selling features.

the_doctah2• 2 years ago

Did the one guy wear a leather jacket so the AI wouldn't point out that he's balding?

hackerlight7• 2 years ago

So what's the difference between the different gpt2 chatbots on lmsys. Which one is deployed live now?

danans• 2 years ago

The sentence order of the Arabic and Urdu examples text is scrambled on that page:

Arabic: مرحبًا، اسمي جي بي تي-4o. أنا نوع جديد من نموذج اللغة، سررت بلقائك!

Urdu: ہیلو، میرا نام جی پی ٹی-4o ہے۔ میں ایک نئے قسم کا زبان ماڈل ہوں، آپ سے مل کر اچھا لگا!

Even if you don't read Arabic or Urdu script, note that the 4 and o are on opposite sides of a sentence. Despite that, pasting both into Google translate actually fixes the error during translation. OpenAI ought to invest in some proofreaders for multilingual blog posts.

tompetry• 2 years ago

I've worked quite a bit with STT and TTS over the past ~7 years, and this is the most impressive and even startling demo I've seen.

But I would like to see how this is integrated into applications by third party developers where the AI is doing a specific job. Is it still as impressive?

The biggest challenge I've had with building any autonomous "agents" with generic LLM's is they are overly gullible and accommodating, requiring the need to revert back to legacy chatbot logic trees etc. to stay on task and perform a job. Also STT is rife with speaker interjections, leading to significant user frustrations and they just want to talk to a person. Hard to see if this is really solved yet.

steve_adams_863• 2 years ago

I’ve found using logic trees with LLMs isn’t necessarily a problem or a deficit. I suppose if they were truly magical and could intuit the right response every time, cool, but I’d always worry about the potential for error and hallucinations.

I’ve found that you can create declarative logic trees from JSON and use that as a prompt for the LLM, which it can then use to traverse the tree accordingly. The only issue I’ve encountered is when it wants to jump to part of the tree which is invalid in the current state. For example, you want to move a user into a flow where certain input is required, but the input hasn’t been provided yet. A transition is suggested to the program by the LLM, but it’s impossible so the LLM has to be prompted that the transition is invalid and to correct itself. If it fails to transition again, a default fallback can be given but it’s not ideal at all.

However, another nice aspect of having the tree declared in advance is that it shows human beings what the system is capable and how it’s intended to be used as well. This has proven to be pretty useful, as letting the LLM call functions it sees fit based on broad intentions and system capabilities leaves humans in the dark a bit.

So, I like the structure and dependability. Maybe one day we can depend on LLM magic and not worry about a team understanding the ins and outs of what should or shouldn’t be possible, but we don’t seem to be there yet at all. That could be in part because my prompts were bad, though.

tomhallett• 2 years ago

Any recommendations on patterns/approaches for these declarative logic trees and where you put which types of logic (logic which goes in the prompt, logic which goes in the code which parses the prompt response, how to detect errors in the response and retry the prompt, etc). On "Show HN" I see a lot of "fully automated agents" which seem interesting, but not sure if they are over-kill or not.

famouswaffles24• 2 years ago

Also STT is rife with speaker interjections, leading to significant user frustrations and they just want to talk to a person. Hard to see if this is really solved yet.

This is not using TTS or STT. Audio and Image data can be tokenized as readily as text. This is simply a LLM that happens to have been trained to receive and spit out audio and image tokens as well as text tokens. Interjections are a lot more palatable in this paradigm as most of the demos show.

somenameforme8• 2 years ago

Adding audio data as a token, in and of itself, would dramatically increase training size, cost, and time for very little benefit. Neural networks also generally tend to function less effectively with highly correlated inputs, which I can only assume is still an issue for LLMs. And adding combined audio training would introduce rather large scale correlations in the inputs.

I would wager like 100:1 that this is just introducing some TTS/STT layers. The video processing layer is probably also doing something similarly, by taking an extremely limited number of 'screenshots', carrying out typical image captioning using another layer, and then feeding that as an input. So the demo, to me, seems most likely to just be 3 separate 'plugins' operating in unison - text to speech, speech to text, and image to text.

The interjections are likely just the software being programmed to aggressively begin output following any lull after an input pattern. Note in basically all the videos, the speakers have to repeatedly cut off the LLM as it starts speaking in conversationally inappropriate locations. In the main video which is just an extremely superficial interaction, the speaker made sure to be constantly speaking when interacting, only pausing once to take a breath that I noticed. He also struggled with the timing of his own responses as the LLM still seems to be attached to its typical, and frequently inappropriate, rambling verbosity (though perhaps I'm not one to critique that).

stitched2gethr• 2 years ago

I would ask you to watch the demo on SoundHound.com. It does less, yes, but it's so crucially fit for use. You'll notice from the shown gpt-4 demo they were guiding the LLM into chain of reasoning. It works very well when you know how to work it, which aligns with what you're saying. I don't mean to degrade the achievement, it's great, but we often inflate the expectations of what something can actually do before reaching real productivity.

Mystery-Machine• 2 years ago

Have you seen this video from Microsoft, uploaded to YT in 2012, the actual video could be even older: https://www.youtube.com/watch?v=Nu-nlQqFCKg

KoolKat233• 2 years ago

I think if you listen to the way it answers, it seems its using a technique trained speakers use. To buy itself time to think, it repeats/paraphrases the question/request before actually answering.

I'm sure you'll find this part is a lot quicker to process, giving the instant response (the old gpt4-turbo is generally very quick with simple requests like this). Rather impressively all it would need is an additional custom instruction.

Very clever and eerily human.

dolmen3• 2 years ago

This behavior is clearly shown on the dad joke demo: https://vimeo.com/945587876

bambax2• 2 years ago

We recognize that GPT-4o’s audio modalities present a variety of novel risks. Today we are publicly releasing text and image inputs and text outputs. Over the upcoming weeks and months, we’ll be working on the technical infrastructure, usability via post-training, and safety necessary to release the other modalities.

So they're using the same GPT4 model with a relatively small improvement, and no voice whatsoever outside of the prerecorded demos. This is not a "launch" or even an announcement. This is a demo of something which may or may not work in the future.

chipgap98• 2 years ago

Better performance, twice the speed, and half the cost is a pretty big win. Demoing the voice features is what makes it an announcement

mucle63• 2 years ago

I just used the voice on my phone after switching to gpt4o and clicking the headphones in the bottom right on iphone

anotherpaulg2• 2 years ago

GPT-4o tops the aider LLM code editing leaderboard at 72.9%, versus 68.4% for Opus. GPT-4o takes second on aider’s refactoring leaderboard with 62.9%, versus Opus at 72.3%.

GPT-4o did much better than the 4-turbo models, and seems much less lazy.

The latest release of aider uses GPT-4o by default.

https://aider.chat/docs/leaderboards/

mucle63• 2 years ago

How am I just hearing about this?! Aider looks cool

AUDI_GUZZ• 2 years ago

this is really amazing!!

pama• 2 years ago

It is absolutely amazing. Thank you to everyone at OpenAI!

pyuser583• 2 years ago

I’m really not impressed.

My academic background is in a field where there are lots of public misconceptions.

It does an absolutely terrible job.

Even basic textbook things where there isn’t much public misunderstanding are “rounded” to what sounds smart.

keroro• 2 years ago

What field? Curious to see it myself

eutropia3• 2 years ago

I mean, it was trained on public internet discourse, probably a bunch of youtube videos, and some legally-grey (thanks copyright) textbooks.

Your field sounds like "There are dozens of us! Dozens!" - who probably all chat at small conferences or correspond through email or academic publication.

Perhaps if it had at its disposal the academic papers, some of the foundational historic documents of record, your emails, textbooks, etc - in a RAG system, or if it had been included in the training corpus it could impress you about this incredibly niche topic.

That said, because it's an ~LLM - its whole thing is generating plausible tokens. I don't know how much work has been put in on an agent level (around or in the primary model) to evaluate confidence on those tokens and hedge the responses accordingly. I doubt it has an explicit notion like some people do of 'hey, this piece of information (<set of coordinates in high dimensional vector space>) [factoid about late ancient egypt] is knowable/falsifiable - and falls under the domain of specialist knowledge: my immense commonsense knowledge might be overconfident given the prevalence of misconceptions in common discourse and I should discount my token probabilities accordingly'

It reflects its training. If there are a lot of public misconceptions, it will have them. Just like most people who are not <expert in arcane academic subtopic>.

dingclancy2• 2 years ago

This is the first demo where you can really sense that beating LLM benchmarks should not be the target. Just remember the time when the iPhone has meager specs but ultimately delivered a better phone experience than the competition.

This is the power of the model where you can own the whole stack and build a product. Open Source will focus on LLM benchmarks since that is the only way foundational models can differentiate themselves, but it does not mean it is a path to a great user experience.

So Open Source models like Llama will be here to stay, but it feels more like if you want to build a compelling product, you have to own and control your own model.

Isinlor• 2 years ago

Depends on the benchmarks. AI that can actually do end to end the job of software developers, theoretical computer scientists, mathematicians etc. would be significantly more impactful than this.

I want to see AI moving the state of the art of the world understanding - physics, mathematics etc. - the way it moved state of the art of the Go game understanding.

dingclancy2• 2 years ago

Doing these end to end jobs still falls on user experience and UI, if we are talking about getting to mass market.

This GPT-4o model is a classic example. It is essentially the same model as GPT-4 but these multimodal features, voice conversations, math, and speed is revolutionary as the creation of the model itself.

Open Source LLM will end up as a model in GitHub and will be used by developers but it looks like even if GPT-4o is only 3 months ahead of other models in terms of benchmarks, the UI + Usecase + Model is 2 years ahead of the competition. And I say that because there is still no chat product that is close to what ChatGPT is delivering now, even though there are models that is close to ChatGPT 4o today.

So if it is sticky for 2 more years, their lead will just grow and we will just end up with more open source models that are technically behind by 3 months but behind product-wise by 2 years.

Adrig• 2 years ago

OpenAI blew up when they released ChatGPT. It was more of a UX breakthrough than pure tech, since GPT3 was available for a few months already.

This feels similar, with OpenAI trying to put their product even more into the daily lives of their users. With GPT4 being good enough for nearly all basic tasks, the natural language and multimodality could be big.

barfingclouds2• 2 years ago

I don’t think Llama being open sourced means Meta has lost anything. If anything it’s just a way to get free community contribution, like Chrome from Chromium. Mega absolutely intends to integrate their version of Llama in their products not so unlike how OpenAI is creating uses for their LLM beyond just the technology

siliconc0w• 2 years ago

I much prefer a GLADOS-type AI voice than one that approximates an endlessly happy chipper enthusiastic personal assistant. I think the AI tutor is probably the strongest for actual real-world value delivered the rest of them are cool but a bit questionable as far as actual pragmatic usefulness.

It'd be cool if an AI calling the another AI would recognize it'd talking to an AI and then they agree to ditch the fake conversational tone and just shift into a high-bandwidth modem pitch to rapidly exchange information. Or upgradable offensive capabilities to outmaneuver the customer service agents when they try to decline your warranty or whatever.

uyzstvqs• 2 years ago

Or upgradable offensive capabilities to outmaneuver the customer service agents when they try to decline your warranty or whatever.

Yeah, OpenAI is not going to do that out of fear of liability. But that's where open source LLM projects will come into play, eg Dolphin.

1vuio0pswjnm7• 2 years ago

OpenAI keeps a copy of all conversations? Or mines them for commercially-useful information?

Has OpenAI found a business model yet. Considering the high cost of the computation, is it reasonable to expect that OpenAI licensing may not be profitable. Will that result in "free" access for the purpose of surveillance and data collection.

Amazon had a microphone in peoples' living rooms, a so-called "smart speaker" that to which people could talk. The "Alexa" was a commercial failure.

nostrebored• 2 years ago

I don’t know anything about conversation retention, but I do know that OpenAI doesn’t plan to be profitable until AGI.

cooper_ganglia2• 2 years ago

As far as I'm aware, anything you input is used as training data unless you have an Enterprise account.

https://help.openai.com/en/articles/7730893-data-controls-fa...

replwoacause2• 2 years ago

Not true. You can opt out using a form they provide, which says they will stop using your data to train the model. I’ve done this. Don’t have the link handy now but it’s not difficult to find.

ein0p2• 2 years ago

What especially blows my mind is not GPT4o. It's that:

1. Nobody could convincingly beat GPT4 in over a year, despite spending billions of dollars trying.

2. There's GPT5 coming out sometime soon that will blow this out of the water and make paying $20/mo to OpenAI still worthwhile.

schappim2• 2 years ago

More accurately, it's impressive that Microsoft, through OpenAI, has stayed ahead of Google, AWS, and Apple while adding $1 trillion to its market cap.

I wouldn't have predicted that it would play out this way.

ein0p2• 2 years ago

You're misplacing the value in this value chain. Without OpenAI Microsoft wouldn't even be in the running. It'd be a second rate cloud provider with a dying office productivity software business. OpenAI, on the other hand, would easily find another company to fund its research.

StarlaAtNight• 2 years ago

These AI's sure do yap a lot

Also, they're TERRIBLE at harmonizing together

fnetisma• 2 years ago

What would be the difference in compute for inference on an audio<>audio model like this compared to a text<>text model?

xarope2• 2 years ago

I'm surprised nobody has mentioned, but this is like shades of the universal translator from star trek.

We have tricorders now (mobile phones), universal translators in the looming... when is transporter technology going to get here?

cooper_ganglia2• 2 years ago

Disney's HoloTile floor feels like the first step to anything resembling a real-life Holodeck...

captaincrunch• 2 years ago

I tried this for about 10 minutes, and went back to 4. Not really that great for what I am doing.

boppo1• 2 years ago

Let's start a betting pool on how long it takes BetterHelp to lay off their therapists for this thing.

timetraveller26• 2 years ago

Just over 10 years later, it's Her

andsoitis3• 2 years ago

what does the "o" stand for?

andsoitis3• 2 years ago

Stands for "omni", since it is multimodal. Source: https://www.tomsguide.com/ai/chatgpt/gpt-4o-is-openais-excit...

stevetron• 2 years ago

I would have liked to see a version number in the prompt, or maybe even have some toggle in my settings, so that I can be certain that I am using ChatGPT 3.5 and then, if I need an image or screen shot analized, I can switch to the limited 4o model. Having my limited availability of 4-o be what gets used, and then not being available becuase of some arbitrary quote that I had no idea was being used-up, is unconscionable policy. Also having no links to email them that fact is bad, too.

booleandilemma• 2 years ago

It's quite scary, honestly. In fact I can't remember the last time a demo terrified me, besides slaughterbots, and that was fictional. I just think about all the possibilities for misuse.

mysore• 2 years ago

does this make retell.ai obsolete?

plaidfuji2• 2 years ago

This is a very cool demo - if you dig deeper there’s a clip of them having a “blind” AI talk to another AI with live camera input to ask it to explain what it’s seeing. Then they, together, sing a song about what they’re looking at, alternating each line, and rhyming with one another. Given all of the isolated capabilities of AI, this isn’t particularly surprising, but seeing it all work together in real time is pretty incredible.

But it’s not scary. It’s… marvelous, cringey, uncomfortable, awe-inspiring. What’s scary is not what AI can currently do, but what we expect from it. Can it do math yet? Can it play chess? Can it write entire apps from scratch? Can it just do my entire job for me?

We’re moving toward a world where every job will be modeled, and you’ll either be an AI owner, a model architect, an agent/hardware engineer, a technician, or just.. training data.

p1esk2• 2 years ago

This is still gpt4. I don’t expect much more from this version than what previous version could do, in terms of reasoning abilities.

But everyone is expecting them to release gpt5 later this year, and it is a bit scary to think what it will be able to do.

jiggawatts3• 2 years ago

Branding aside, this pretty much is GPT 5.

The evidence for that is the change in the tokenizer. The only way to implement that is to re-train the entire base model from scratch. This implies that GPT 4o is not a fine-tuning of GPT 4. It's a new model, with a new tokenizer, new input and output token types, etc...

They could have called it GPT-5 and everyone would have believed them.

golol9• 2 years ago

I think 4o is actually noticeably smarter than 4, after having tried it a tiny bit on the playground.

trashtester15• 2 years ago

It's quite different from gpt4 in two respects:

1) It's natively multi-modal in a way I don't think gpt4 was.

2) It's at least twice as efficient in terms of compute. Maybe 3 times more efficient, considering the increase in performance.

Combined, those point towards some major breakthroughs having gone into the model. If the quality of the output hasn't gone up THAT much, it's probably because the technological innovations mostly were leveraged (for this version) to reduce costs rather than capabilities.

My guess is that we should expect them to leverage the 2x-3x boost in efficiency in a model that is at least as large as GTP4 relatively soon, probably this year unless OpenAI has safety concerns or something, and keeps it internal-only.

hackerlight7• 2 years ago

It's a completely new model trained from scratch that they've decided to label that way as part of their marketing strategy.

anon3738394• 2 years ago

We’re moving toward a world where every job will be modeled

After an OpenAI launch, I think it's important to take one's feelings about the future impact of the technology with a HUGE grain of salt. OpenAI are masters of hype. They have been generating hype for years now, yet the real-world impacts remain modest so far.

Do you remember when they teased GPT-2 as "too dangerous" for public access? I do. Yet we now have Llama 3 in the wild, which even at the smaller 8B size is about as powerful as the [edit: 6/13/23] GPT-4 release.

As someone pointed out elsewhere in the comments, a logistic curve looks exponential in the beginning, before it approaches saturation. Yet, logistic curves are more common, especially in ML. I think it's interesting that GPT-4o doesn't show much of an improvement in "reasoning" strength.

owenpalmer• 2 years ago

They're masters of hype because their products speak for themselves (literally)

danielscrubs2• 2 years ago

"real-world impacts remain modest so far." Really? My Google usage has went down with 90% (it would just lead me to some really bad take from a journalist anyway, while ChatGPT can just hand me the latest research and knows my level of expertise). Sure it is not so helpful at work, but if OpenAI hasnt impacted the world I fail to see which company have in this decade.

kenjackson3• 2 years ago

GPT-4 already seems better at reasoning than most people. It just has an unusual training domain of Internet text.

jdietrich5• 2 years ago

A Google search for practically any long-tail keywords will reveal that LLMs have already had a very significant impact. DuckDuckGo has suffered even more. Social media is absolutely lousy with AI-powered fraud of varying degrees of sophistication.

It's glib to dismiss safety concerns because we haven't all turned into paperclips yet. LLMs and image gen models are having real effects now.

We're already at a point where AI can generate text and images that will fool a lot of people a lot of the time. For every college-educated young person smugly pointing out that they aren't fooled by an image with six-fingered hands, there are far more people who had marginal media literacy to begin with and are now almost defenceless against a tidal wave of hyper-scaleable deception.

We're already at a point where we're counselling elders to ignore late-night messages from people claiming to be a relative in need of an urgent wire transfer. What defences do we have when an LLM will be able to have a completely fluent, natural-sounding conversation in someone else's voice? I'm not confident that I'd be able to distinguish GPT-4o from a human speaker in the best of circumstances and I'm almost certain that I could be fooled if I'm hurried, distracted, sleep deprived or otherwise impaired.

Regardless of any future impacts on the labour market or any hypothesised X-risks, I think we should be very worried about the immediate risks to trust and social cohesion. An awful lot of people are turning into paranoid weirdos at the moment and I don't particularly blame them, but I can see things getting seriously ugly if we can't abate that trend.

lynx232• 2 years ago

yet the real-world impacts remain modest so far.

I spend a part of yesterday evening sorting my freshly dried t-shirts into 4 distinct piles. I used OpenAI Vision (through BeMyEyes) from my phone. I got a clear description of each and every piece of clothing, including print, colours and brand. I am blind BTW. But I guess you are right, no impact at all.

Yet we now have Llama 3 in the wild

Yes, great, THANKS Meta, now the Scammers have something to work with. Thats a wonderful achievement which should be praised! </sarcasm>

fhe• 2 years ago

yet the real-world impacts remain modest so far.

I second that. I remember when Google search first came out. Within a few days it completely changed my workflow, how I use the Internet, my reading habits. It easily 5 ~ 10x the value of Internet for me over a couple of weeks.

LLMs is doing nothing of the sort for me.

koonsolo• 2 years ago

yet the real-world impacts remain modest so far

Go ask any teacher or graphician.

KronisLV• 2 years ago

Do you remember when they teased GPT-2 as "too dangerous" for public access? I do.

Maybe not GPT-2, but in general LLMs and other generative AI types aren't without their downsides.

From companies looking to downsize their staff to replace them with software, to the work of artists/writers being devalued somewhat, to even easier scams and something like the rise of AI girlfriends, which has also gotten some critique, some of those can probably be a net negative.

Even when it's not pearl clutching over the advancements in technology and the social changes that arise, I do wonder how much my own development work will be devalued due to the somewhat lowered entry barrier into the industry and people looking for quick cash, same as with boot camps leading to more saturation. Probably not my position individually (not exactly entry level), but the market as a whole.

It's kind of at a point where I use LLMs for dev work not to fall behind, cause the productivity gains for simple problems and boilerplate are hard to argue with.

ben_w12• 2 years ago

OpenAI are masters of hype. They have been generating hype for years now, yet the real-world impacts remain modest so far.

Perhaps.

Do you remember when they teased GPT-2 as "too dangerous" for public access? I do. Yet we now have Llama 3 in the wild, which even at the smaller 8B size is about as powerful as the [edit: 6/13/23] GPT-4 release.

The statement was rather more prosaic and less surprising; are you sure it's OpenAI (rather than say all the AI fans and the press) who are hyping?

"""This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas.

…

We are aware that some researchers have the technical capacity to reproduce and open source our results. We believe our release strategy limits the initial set of organizations who may choose to do this, and gives the AI community more time to have a discussion about the implications of such systems."""

keiferski4• 2 years ago

I can’t even get GPT 4 to reliably take a list of data and put it in a CSV. It gets a problem every single time.

People read too many sci-fi books and then project their fantasies on to real-world technologies. This stuff is incredibly powerful and will have social effects, but it’s not going to replace every single job by next year.

jstummbillig3• 2 years ago

Do you remember when they teased GPT-2 as "too dangerous" for public access? I do.

I can't help but notice the huge amount of hindsight and bad faith that it demonstrated here. Yes, now we are aware that the internet did not drown in a flood of bullshit (well, not noticeably more), when GPT-2 was released.

But was it obvious? I certainly thought that there was a chance that the amount of blog spam that could be generated effortlessly might just make internet search unusable. You are declaring "hype", when you could also say "very uncertain and conscientious". Is this not something we want people in charge to be careful with?

naasking3• 2 years ago

They have been generating hype for years now, yet the real-world impacts remain modest so far.

I feel like everyone who makes this claim doesn't actually have any data to backup it up.

collyw• 2 years ago

Chat GPT 3.5 has been neutered, as it it won't spit out anything that isn't overly politically correct. 4chan were hacking their way around it. Maybe that's why they decided it was "too dangerous".

FrustratedMonky• 2 years ago

" GPT-4o doesn't show much of an improvement in "reasoning" strength."

Maybe that is GPT-5.

And this release really is just incremental improvements in speed, and tying together a few different existing features.

doug_durham• 2 years ago

It's well known that LLMs don't reason. That's not what they are for. It's a throw away comment to say that a product can't do what it explicitly is unable to do. Reasoning will require different architectures. Even with that LLMs are incredibly useful.

somenameforme8• 2 years ago

Like another comment mentioned, sigmoid curves [1] are ubiquitous with neural network systems. Neural network systems can be intoxicating because it's so "easy" (relatively speaking) to go from nothing to 80% in extremely short periods of time. And so it seems completely obvious that hitting 100% is imminent. Yet it turns out that each percent afterwards starts coming exponentially more slowly, and we tend to just bump into seemingly impassable asymptotes far from where we'd like to be.

~8 years ago when self driving technology was all the rage and every major company was getting on board with ever more impressive technological demos, it seemed entirely reasonable to expect that we'd all be in a world of complete self driving imminently. I remember mocking somebody online around the time who was pursuing a class C/commercial trucking license. Yet now a decade later, there are more truckers than ever and the tech itself seems further away than ever before. And that's because most have now accepted that progress on such has basically stalled out in spite of absolutely monumental efforts at moving forward.

So long as LLMs regularly hallucinate, they're not going to be useful for much other than tasks that can accept relatively high rates of failure. And many of those generally creative domains are the ones LLMs are paradoxically the weakest in - like writing. Reading a book written by an LLM would be cruel and unusual punishment given then current state of the art. One domain I do see them completely taking over is search. They work excellently as natural language search engines, and "failure" in such is very poorly defined.

[1] https://en.wikipedia.org/wiki/Sigmoid_function

mlsu2• 2 years ago

I'm not really sure your self-driving analogy is apt here. Waymo has cars on the road right now that are totally autonomous, and just expanded its footprint. It has been longer and more difficult than we all thought, and those early tech demos were a glimmer of what was to come; then we had to grind to get there, with a lot of engineering.

I think what maybe seems not obvious amidst the hype is that there is a hell of a lot of engineering left to do. The fact that you can squash the weights of a neural net down to 3 bits per param and it still works -- is evidence that we have quite a way to go with maturing this technology. Multimodality, improvements to the UX of it, the human-computer interface part of it. Those are fundamental tech things, but they are foremost engineering problems. Getting latency down. Getting efficiency up. Designing the experience, then building it out.

25 years ago, early tech demos on the internet were promising that everyone would do their shopping, entertainment, socializing, etc... online. Breathless hype. 5 years after that, the whole thing crashed, but it never went away. People just needed time to figure out how to use it and what it was useful for, and discover its limitations. 10 years after that, engineering efforts were systematized and applied against the difficult problems that still remained. And now: look at where we are. It just took time.

rossant• 2 years ago

It's been 8 years and I still don't have my autonomous car.

Meanwhile I've been using ChatGPT at work for _more than a year_ and it's been tremendously helpful to me.

This is not hype, this is not about how AI will change our lives in the future. It's there right here, right now.

jstummbillig3• 2 years ago

So long as LLMs regularly hallucinate, they're not going to be useful for much other than tasks that can accept relatively high rates of failure.

But is this not what humans do, universally? We are certainly good at hiding it – and we are all good at coping with it – but my general sense when interacting with society is that there is a large amount of nonsense generated by humans that our systems must and do already have enormous flexibility for.

My sense is that's not an aspect of LLMs we should have any trouble with incorporating smoothly, just by adhering to the safety nets that we built in response to our own deficiencies.

davedx• 2 years ago

So long as LLMs regularly hallucinate, they're not going to be useful for much other than tasks that can accept relatively high rates of failure.

Yep. So basically they're useful for a vast, immense range of tasks today.

Some things they're not suited for. For example, I've been working on a system to extract certain financial "facts" across SEC filings. ChatGPT has not been helpful at all either with designing or implementing (except to give some broad, obvious hints about things like regular expressions), nor would it be useful if it was used for the actual automation.

But for many, many other tasks -- like design, architecture, brainstorming, marketing, sales, summarisation, step by step thinking through all sorts of processes, it's extremely valuable today. My list of ChatGPT sessions is so long already and I can't imagine life without it now. Going back to Google and random Quora/StackOverflow answers laced with adtech everywhere...

nox1012• 2 years ago

I think it's more like an exponential curve where it looks flat moments before it shoots up.

mapping th genome was that way. On a 20yr schedule, barely any progress for 15 and then poof, done ahead of schedule

SubiculumCode3• 2 years ago

The sigmoid is true in humans too. You can get 80% of the way to being sort of good at a thing in a couple of weeks, but then you hit the plateau. In a lot of fields confidently knowing and applying this has made people local jack of all trades experts... the person that often knows how to solve the problem. But Jack is no longer needed so much. ChatJack got`s your back. Better to be a the person who knows one thing in excruciating detail and depth, and never ever let anyone watch you work or train on your output.

Jiahang• 2 years ago

make people smart

starfezzy• 2 years ago

Nature had been doing that for billions of years until a few decades ago when we were told "progress" meant we had to stop doing the same thing more peacefully and intentionally.

My guess is the future belongs to those who don't stop—who, in fact, embrace the opposite of stopping.

I would even suggest that the present belongs to those who didn't stop. It may be too late for normal people to ever catch up by the time we realize the trick that was played on us.

whilenot-dev• 2 years ago

or just.. training data.

I have a much less "utopian" view about the future. I remember during the renaissance of neural networks (ca. 2010-15) it was said that "more data leads to better models", and that was at a time when researchers frowned upon the term Artificial Intelligence and would rather use Machine Learning. Fast forward a decade LLMs are very good synthetic data generators that try to mimic human generated input and I can't think somehow that this wasn't the sole initial intent of LLMs. And that's it for me. There's not much to hype and no intelligence at all.

What happens now is that human generated input becomes more valuable and every online platform (including minor ones) will have now some form of gatekeeping in place, rather sooner than later. Besides that a lot of work still can't be done in front of a computer in isolation and probably never will, and even if so, automation is not a means to an end. We still don't know how to measure a lot of things and much less how to capture everything as data vectors.

aussieguy12342• 2 years ago

My new PC arrives tomorrow. Once I source myself two RTX 3060's I'll be an AI owner, no longer dependant on cloud APIs.

Currently the bottleneck is Agents. If you want a large language model to actually do anything you need an Agent. Agents so far need a human in the loop to keep them sane. Until that problem is solved most human jobs are still safe.

sambazi• 2 years ago

Agents so far need a human in the loop to keep them sane.

not quite sure that sanity is a business requirement

trashtester15• 2 years ago

GPT 4o incorporated multimodality directly in the neural network, while reducing inference costs to half.

I fully expect GPT 5 (or at the latest 6) to similarly have native inclusion of agentic capabilities either this year or next year, assuming it doesn't already, but is just kept from the public.

selfhoster11• 2 years ago

You are far better off investing in one or more 3090s and loading up on DDR RAM.

megous• 2 years ago

Why so much positivity? It can also murder people, and it will continue being used for that. That's scary.

ithkuil• 2 years ago

Probably it will be used for many things. Just like computers, electricity, iron

onion2k• 2 years ago

(IMO) AI cannot murder people. The responsibility of what an AI does falls on the person who deployed it, and to a lesser extent the person who created it. If someone is killed by a fully autonomous weapon then that person has been murdered by the person or people who created and enabled the AI, not the AI itself.

This is no different to saying a person with a gun murdered someone rather than attributing the murder to the gun. An AI gun is just a really fancy gun.

kolinko3• 2 years ago

So can knives

mavhc• 2 years ago

Don't need AI to murder people, a gun with an actuator on the trigger can murder people easily, add rnd to it and it can murder people at random

interludead5• 2 years ago

Anything (almost) can be used for good and bad

aksss2• 2 years ago

Or a battery.

datahack• 2 years ago

All I could think about when watching this demo was how similar capabilities will work on the battlefield. Coordinated AIs look like they will be obscenely effective.

Everything always starts as a toy.

trashtester15• 2 years ago

The "Killer app" for AGI/ASI is, I suspect, going to be in robotics, even more so than in replacing "white collar workers".

That includes, beyond literal Killers, all kinds of manufacturing, construction and service work.

I would expect a LOT of funds to go into research all sorts of actuators, artificial muscles and any other technology that will be useful in building better robots.

Companies that can get and maintain a lead in such technologies may reach a position similar to what US Steel had in the 19th century.

That could be the next nvidia.

I would not be at all surprised if we will have a robot in the house in 10 years that can clean and do the dishes, and that is built using basically the same parts as the robots that replace our soldiers and the police.

Who will ultimately control them, though?

hwbunny2• 2 years ago

And people will become utterly stupid in the process.

CTDOCodebases• 2 years ago

or just... unemployed.

visarga3• 2 years ago

Can it just do my entire job for me?

All AIs up to now lack autonomy. So I'd say until we crack this problem, it is not going to be able to do your job. Autonomy depends on a kind of data that is iterative, multi-turn, and learning from environments not from static datasets. We have the exact opposite, lots of non-iterative, off-policy (human made AI consumed) text.

schnitzelstoat• 2 years ago

Until the hallucination problem is solved, the output can't be trusted.

So outside of use-cases where the user can quickly verify the result (like picking a decent generated image etc.),I can't see it being used much.

zarathustreal2• 2 years ago

Never heard of retrieval-augmented generation?

mottiden2• 2 years ago

We’re moving toward a world where every job will be modeled, and you’ll either be an AI owner, a model architect, an agent/hardware engineer, a technician, or just.. training data.

I understand that you might be afraid. I believe that a world where only LLM companies rule the world is not practically achievable except in some distopian universe. The likelihood of the world where the only job are model architects, engineers or technicians is very very small.

Instead, let's consider the positive possibilities that LLMs can bring. It can lead to new and exciting opportunities across various fields. For instance, can serve as a tool to inspire new ideas for writers, artists, and musicians.

I think we are going towards a more collaborative era where computers and humans interact much more. Everything will be a remix :)

altcognito2• 2 years ago

The likelihood of the world where the only job are model architects, engineers or technicians is very very small.

Oh, especially since it will be a priority to automate their jobs, or somehow optimize them with an algorithm because that's a self-reinforcing improvement scheme that would give you a huge edge.

Aeolun3• 2 years ago

The two AI’s talking to each other was like listening to two commercials talking to each other. Like a callcenter menu that you cannot skip. And they _kept repeating themselves_. Ugh. If this is the future I’m going to hide in a cave.

mycall2• 2 years ago

Can it do math yet?

GPT-4? Not that well. AI? Definitely

https://deepmind.google/discover/blog/alphageometry-an-olymp...

aredox• 2 years ago

Call me back when we have autonomous driving, and when Bitcoin will replace currencies.

What's scary and cringey are your delusions.

plaidfuji2• 2 years ago

Don’t get me wrong, I’m not suggesting the current capabilities are anywhere near replacing human productivity. Some things are 1 year out, some 5 (maybe self-driving cars by then? Mercedes has it on their roadmap for 2030 and they’ve historically been realistic), some 10+. But the pieces are in place and the investments are being made. The question is no longer “can AI really automate this?”, it’s “how do we get the dataset that will enable us to automate this with AI?”. And as long as Open AI keeps people’s eyes on their whizbang demos, the money will keep flowing…

giuscri• 2 years ago

Can someone explain how you can interrupt with the voice this model? Where do I read more technical details about this?

no7hing• 2 years ago

Does anyone have technical insight into how the screensharing in the math tutor video works? It looks like they start the broadcast from within the ChatGPT app, yet have no option to select which app will be the source of the stream. Or is that implied when both apps reside in the iPad's split view? And is this using regular ReplayKit or something new?

doku• 2 years ago

FREE = Data Collection

radicality2• 2 years ago

Am I using it wrong? I have the gpt plus subscription, and can select "gpt4o" from the model list on ChatGPT, but whichever example I try from the example list under "Explorations of capabilities" on `https://openai.com/index/hello-gpt-4o/`, my results are worse:

* "Poetic typography" sample: I paste the prompt, and get an image with the typical lack of coherent text, just mangled letters.

* "Visual Narratives: Robot Writer's Block" - Mangled letters also

* "Visual Narratives: Sally the mailwoman" - not following instructions about camera angle. Sally looks different in each subsequent photo.

* "Meeting Notes with multiple speakers" - I uploaded the exact same audio file and used input 'How many speakers in this audio and what happened?'. gpt4o went off about about audio sample rates, speaker diarization models, torchaudio, and how its execution environment is broken and can't proceed to do it.

fragmede• 2 years ago

link the chat?

atleastoptimal• 2 years ago

they haven’t released gpt-4o image capabilities yet, it defaults to dalle 3

radicality2• 2 years ago

Ah, I see. Seems like a weird product release? Since everything in the UI (and in the new ChatGPT macos app) says 'gpt4o' so I would expect at least something to work as shown in the demos. Or just don't show at all the 'gpt4o' in the UI if it's somehow a completely different 'gpt4o' from the one that can do everything on the announcements page. I don't mind waiting, but it was genuinely confusing to me.

antman• 2 years ago

Looking forward to see how one can finetune this

chilling3• 2 years ago

The similiarity between this model and the movie 'Her' [0] creeps me out so badly that I can't shake the feeling that our social interactions are on the brink of doom.

[0] https://youtu.be/GV01B5kVsC0?feature=shared&t=85

radres2• 2 years ago

it is so similar! almost it is inspired by "Her"

matesz• 2 years ago

Don't worry. "Her", in its own right, is frightening, but this is because there is no transparency - you actually can't see how it works and you can't personalize it - choose different options.

Once you grasp that, at least this level of fear should go away. Of course, I'm sure there are more levels of fear related to AI :) just don't have enough time to think about it, perhaps good for me.

osigurdson3• 2 years ago

I appear to have GPT-4o but the iPhone app seems to be largely the same - can't interrupt it, no "emotive" voice, etc. Is this expected?

dmje• 2 years ago

Being sarcastic and then putting the end result in front of Brits could be the new Turing Test

p1dda• 2 years ago

First impressions as a 1-year subscriber. I just tried GTP-4o to evaluate my code for suggestions and for discussing other solutions and it is definitely faster and it comes up with new suggestions that GPT-4 didn't. Currently in the process of evaluating the suggestions.

The demo is what it is, designed to get a wow from the masses.

callen43• 2 years ago

This is clearly not just another story of human innovation. This is not just the usual trade-off between risks and opportunities.

Why? Because it simply automates the human away. Who wouldn't opt for a seemingly flawless, super effective buddy (i.e. an AI) that is never tired, always knows better? if you need some job done, if you're feeling lonely, when you need some life advice.. It doesn't matter if it might be considered "just imitation of human".

Why would future advancements of it keep being "just some tool" instead of largely replacing us as (humans) in jobs, relationships, ...?

tonyabracadabra• 2 years ago

wild https://gpt4o.ai

TheAceOfHearts• 2 years ago

I wish the presentation had included an example of integration with a simple tool like a timer. Being able to set and dismiss a timer in casual conversation while multitasking would be a really great demo of integrated capabilities.

DeathArrow• 2 years ago

I am not impressed. We already have better models for text to peach and voice synthetization. What we see here is integration with a LLM. One can do it at home by combining Llama3 with text to speech and voice synth.

What would amaze me would be for GPT 4 to have better reasoning capabilities and less hallucinations.

latentsea2• 2 years ago

Is this actually available in the app in the same way they are demoing it here? I see the model is available to be selected, but the interface doesn't quite seem to allow me to use it in the way I see here.

Giorgi• 2 years ago

and no mention of hallucinations. I hope it was improved.

sholladay• 2 years ago

The emphasis on multimodal made me wonder if it was capable of creating audio as output, so I asked it to make me a drum beat. It did so, but in text form. I asked it to convert it to audio. It thought for a while and eventually said it didn’t seem like `simpleaudio` was installed in its environment. Huh, interesting, never seen a response like that before. It clearly made an attempt to carry out my instructions but failed due to technical limitations of its backend. What else can I make it do? I asked it to install `simpleaudio`. It tried but failed with a connection error, presumably due to a firewall rule.

I asked it to run a loop that writes “hello” every ten seconds. Wow, not only did it do so, it’s streaming the stdout to me.

LLMs have always had various forms of injection attacks, ways to force them to reveal their prompts, etc. but this one seems deliberately designed to run arbitrary code, including infinite loops.

Alas, I doubt I can get it to mine enough bitcoin to pay for a ChatGPT subscription,

https://x.com/sethholladay/status/1790233978290516453

xlbuttplug2• 2 years ago

people are scared and it shows :)

dynamite-ready• 2 years ago

At their core, I still think of these things as search engines, albeit super advanced ones. But the emotion the agent conveys with it's speech synth is completely new...

dlimeng• 2 years ago

After looking at the introduction, there doesn't seem to be much of an update in OpenAI's features: https://aidisruption.substack.com/p/ultimate-ai-gpt-4o-your-...

maaaaattttt2• 2 years ago

Now that I see this, here is my wish (I know there are security privacy concerns but let's pretend there are not there for this wish): An app that runs on my desktop and has access to my screen(s) when I work. At any time I can ask it something about what's on the screen, it can jump in and let me know if it thinks I made a mistake (think pair programming) or a suggestion (drafting a document). It can also quickly take over if I ask it too (copilot on demand).

Except for the last point and the desktop version I think it's already done in math demo video.

I guess it will also pretty soon refuse to let me come back inside the spaceship, but until then it'll be a nice ride.

ideamotor• 2 years ago

This basically already exists and the companies that sell this are constantly improving it. For better or worse.

peddling-brink• 2 years ago

“You haven’t done anything productive for 15 minutes. Are you taking an unauthorized break?”

devonsolomon• 2 years ago

Agreed. I’m excited about reaching a point where the experience is of being in a deep work ‘flow’ with an ultra intelligent colleague, instead of jumping out of context to instant message them.

iLoveOncall2• 2 years ago

with an ultra intelligent colleague

Ultra knowledgeable but pretty stupid actually.

chilling3• 2 years ago

HAL. HAL? HAL!

pkos98• 2 years ago

Here you go: UFO - A UI-Focused Agent for Windows OS Interaction

"UFO is a UI-Focused dual-agent framework to fulfill user requests on Windows OS by seamlessly navigating and operating within individual or spanning multiple applications."

https://github.com/microsoft/UFO?tab=readme-ov-file

lm284695• 2 years ago

1984's telescreens aren't something you're supposed to wish for

maaaaattttt2• 2 years ago

You mean I can't wish for a brave new world?

Joking aside, I agree. It's too bad, though, that we know a thing (this or anything else even technological or not) that could be used for good and improving ourselves will almost always be diverted for something bad...

hbn4• 2 years ago

One of my favorite tweets:

    Sci-Fi Author: In my book I invented the Torment Nexus as a cautionary tale

    Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus

https://twitter.com/AlexBlechman/status/1457842724128833538?...

brap2• 2 years ago

This makes me think, we're seeing all these products inject AI and try to be "smart" on their own, but maybe the experience we really need is a smart OS that can easily orchestrate dumb products.

I know that Siri/Google Assistant/Cortana(?) can already integrate with 3p apps, so maybe something like this but much smarter. e.g. instead of "send the following email" you would tell the assistant "just write the email yourself". At this point your email app doesn't need integrated AI anymore. Just hooks for the assistant.

I imagine once Google puts that kind of brains on Android and Chrome, many product devs will no longer need to use AI directly. Two birds one stone situation, since these devs won't need OpenAI.

geonnave• 2 years ago

Something similar already exists, see https://www.rewind.ai/ and https://www.perfectmemory.ai/

barfingclouds2• 2 years ago

They showed just that in the demo with the voice call example. Screen share can be a live video feed

gloosx• 2 years ago

New flagship... This is becoming to look like a smartphone world, and Sam Altman is a Steve Jobs of this stuff. At some point tech will reach saturation and every next model will be just 10% faster, 2% less hallucination, more megapixels for images etc :)

adroniser3• 2 years ago

Doubtful. Apple can get away with tiny iterations on smartphones because they have the brand and they know people will always buy their latest product. LLMs aren't physical products so there is no cost to switching other than increased API cost, meaning openAI won't be able to recoup the cost of training a new model unless the model is sufficiently different that it justifies people paying significantly more for.

The special thing about GPT-4o is the multimodal capabilities, all the metrics suggest that it is the same size language model roughly as GPT-4. The fact it's available for free also points to it not being the most intelligent model that openAI has atm.

The time to evaluate whether we're starting to level off is when they've trained a model 10x larger than gpt-4 and we don't see significant change.

potet• 2 years ago

\clear

mppm• 2 years ago

Such an impressive demo... but why did they have to give it this vapid, giggly socialite persona that makes me want to switch it off after thirty seconds?

sainez• 2 years ago

You should be able to adjust this with a system prompt, given that has end-to-end speech capabilities now

jeisc• 2 years ago

a fake saucy friend for alienated humans with chitchat

if the end user is in a war zone will the AI bot still ask how it is going?

how many bombs fell in your neighborhood last night?

Fiahil• 2 years ago

Can I mix french and english when talking to it ?

robblbobbl• 2 years ago

Haha lol Subscription canceled was the best choice but it's new fancy cool magic sensational AGI please give all your money df

neeloor2004• 2 years ago

can i try free ?

nbzso• 2 years ago

Another new hit from his excellency Sam the Galactic Conmaster? WoW. The future is bright, right?:)

Idiocracy in full swing, dear Marvin.

digitcatphd• 2 years ago

Hopefully this will be them turning a new leaf. Making GPT-4 more accessible, cutting API costs, and making a general personal assistant chatbot on iPhone are a lot different than them tracking down and destroying the business of every customer using their API one by one. Let's hope this trend continues.

radres2• 2 years ago

What popped out to me in the "bunny ear" video, the bunny ears are not actually visible to the phone's camera Greg is holding. Are they in the background feeding the production camera and this is not really a live demo?

hackerlight7• 2 years ago

The bunny ears were actually visible on the phone's camera for about one second towards the end.

laylower• 2 years ago

Set a memorable verification phrase with your friends and loved ones.

wiz21c2• 2 years ago

That woman's voice intonation is just scary.Not because it talks really well, but because it is always happy, optimistic, enthusiastic. And this echoes to what several of my employers idealized as a good employee.

That's terrifying because those AI become what their master's think an engaging human should be. It's quite close to Bostondynamics di some years ago. what did they show ? You can hit a robot very hard while it does its job and then what ? It just goes on without complaining. A perfect employee again.

That's very dystopic to me.

(but I'm impressed by the technical achievement)

sametmax• 2 years ago

I get the feeling but what altenrative would you prefer?

I do want my phone to just go again without complaining without a crash, after all.

wiz21c2• 2 years ago

dunno tbh. When I hear HAL in 2001, it's more neutral but more scary. I have no idea. that tells a lot.

titzer3• 2 years ago

I want Richard Dawkins or David Attenborough to narrate my code for me.

pquki4• 2 years ago

To be honest, this comment makes me think it would be interesting to have a grumpy version of AI just for fun

crvdgc• 2 years ago

Marvin from The Hitchhikers' Guide to Galaxy.

jstummbillig3• 2 years ago

Absolutely. I can feel but — but this is one/two calibration steps away from me not caring/noticing. I would be very hard pressed to believe this is where the magic human sauce will forever lie (or even more than +1 year), will fully acknowledging how weirdly off it feels right this moment. The progress is undeniably at a speed that surpasses anything I can process or adjust to. It's a rollercoaster ride.

I have no trouble believing the best (whatever that means to me) humans that have ever existed in 2 years will not be human. But I have trouble understanding.

lagrange772• 2 years ago

Yes it feels very cyberpunk.

Me: I can't breathe, am i going to die?

Enthusiastic woman's voice: Yes Jim, that's absolutely correct! You will die of suffocation in approximately 3 minutes 41 seconds. Anything else i can do for you?

drexlspivey• 2 years ago

Me: Open the pod door please

btbuildem• 2 years ago

It'll be more like: "This content is in violation of our policies. Your account will be temporarily disabled for seven days."

jodrellblank4• 2 years ago

Someone else in these comments linked a Unitree biped robot demo video, and that has both someone kicking it in the back and punching it in the chest with a boxing glove on to show that it doesn’t fall over. And nothing else - no neutral trip hazard, opening a door in its way, slippery floor surface, gust of wind - only physical assault from a larger humanoid.

brap2• 2 years ago

The first thought that came to my head was that it's being "way too American". Not in a good way.

aeyes2• 2 years ago

The only thing I'm missing is: "That's a great question!"

I wonder how they beat that out of the model.

BriggyDwiggs42• 2 years ago

Yeah, but honestly at the same time we’ve got useful models with no self awareness. Aside from the exhausting corpo-speak, we’ve got no reason to want anything other than something convenient for us.

gpmcadam• 2 years ago

I think also very US-culture-specific.

henry2023• 2 years ago

Like the most of the content this llm was fed upon?

dionian2• 2 years ago

Her enthusiasm about it "being about her" was really bizarre and I wonder if it wasn't staged a bit. I mean I hope it was staged a bit. If my AI servant started wasting all that time by acting all silly I would be pretty annoyed. But maybe it's smart enough to know that this is a situation where they just want her to be playful and fun instead of deliver information.

latexr6• 2 years ago

You can hit a robot very hard while it does its job and then what ? It just goes on without complaining.

Maybe you’re referring to a different video than the one I watched (or I may be misremembering), but from what I recall the point of the video didn’t seem to be “you can abuse robots and they won’t fight back” but rather to show them recovering well from unpredictable situations (which could be environmental, not human).

Well, there was one video where the point was abuse, but that was CGI and not made by Boston Dynamics.

https://www.theverge.com/tldr/2019/6/17/18681682/boston-dyna...

I do agree with your larger point, I don’t wish to detract from that.

titzer3• 2 years ago

The AI woman's voice is far too breathy and emotive. I instantly hate it and don't want to hear it. The AI has also copied one of my personal pet peeves, which is dropping or swallowing d's and t's from certain words, like "didn't" to "di-unt" and "important" to "impor-unt"--which I find to be a new unbearable laziness spreading amongst younger Americans today. There are TWO T's in important goddammit (I'll die on that hill).

I hate this voice, it will just overprint everyone's voice now with Silicon Valley's annoying "Valley-girl-lite" accent.

davidmurdoch• 2 years ago

The ts and ds thing... it's just language. It changes over time, it's not youth being lazy... The slang of the youth today is actually kinda wordy and extra.

Anyway, I too think today's youth's slang and language is annoying, but not really something the older generations get a say in.

cma• 2 years ago

I'll die on that hill

"I'll" instead of I will? Is it shortcuts you are mad at or just change?

bookaway• 2 years ago

I'm sure they'll add a Interstellar/TARS demo soon:

Chirpiness Level?
Chirpiness at 80%!
Set chirpiness to 10%

[in noticeable monotone]

Chripiness set to 10%.

mrtranscendence3• 2 years ago

Important is pronounced not with two hard t sounds, but with a glottal stop: Impor[glottal stop]ant. That's not laziness, that's my actual dialect of English spoken by actual adult people.

ryandrake2• 2 years ago

It sounds like a cross between the "TikTok text-to-speech valley girl voice" and a "Real Housewives reality-tv voice". The worst of both worlds when it comes to an annoying voice. Why would you pick something like that for what is supposed to represent a helpful assistant?

varispeed• 2 years ago

This could be part of great stress relief device.

Just install it in a mannequin with with a punchable face, telling you how sorry it is that your are struggling with your life, with that happy, ironic and cynical voice intonation.

DenisM• 2 years ago

I believe you might be looking for the term “toxic positivity”? I don’t necessarily subscribe to that description, but I see where you’re coming from.

It’s an interesting choice on their part. Probably targeted at a certain audience who are not grumpy software engineers (myself included).

To be fair, they do have a “sarcastic” voice demo and I found it quite endearing.

I think there will be something for everyone in the end.

sandspar• 2 years ago

I assume you can give voice-specific custom instructions. "Talk like a Clive Cussler audiobook narrator" - problem solved.

rapjr9• 2 years ago

I see a wider problem here, interacting with this AI could train people to behave the same way with real people. Interrupt them whenever you feel like it, order them to do things, turn them off, walk away while they are still talking. People may start imitating the AI's behavior to put someone down, treat them as second class, as though they were also an AI just to be used, and not a person. If people use this conversational AI often then the ways they interact with it will creep into peoples use of language with each other. People imitate each other all the time, they will start imitating the AI. They'll think it is funny but after a while it may not turn out to be so funny.

doubloon2• 2 years ago

Its great tech and i thought i wanted it but…. After talking to it for a few hours i got this really bizarre odd gut feeling of disturbance and discomfort, disconnection from reality. It reminds me of wearing VR goggles. Its not just the physical issues there is something psychologically disturbing about it. It wont even give itself a name. I honestly prefer Siri even though she is incompetent she is “honest” in her incompetence. Also i left the thing on accidentally and it said it had an eight hour chat with me lol

hackerlight7• 2 years ago

Audio hasn't been released yet

doubloon2• 2 years ago

Uhm it has on my account. I had an extended conversation about type design in language

gizajob2• 2 years ago

I, for one, welcome our new annoying-sounding AI overlords.

vbezhenar5• 2 years ago

The more I get, the more I want. Exciting times. Can't wait for GPT-5.

reportgunner2• 2 years ago

Nice, the landing page says "Try on ChatGPT" but when I create an account all I can try is GPT3.5.

I am not surprised.

can16358p• 2 years ago

Am I missing something?

I've picked GPT-4o model in ChatGPT app (I have the paid plan), started talking with the voice mode: both the responses are much slower than in the demo, and there is no way to interrupt the response naturally (I need to tap a button on screen to interrupt), and no way to open up camera and show around like the demo does.

jerpint• 2 years ago

I don’t think the app version is available

d4rkp4ttern• 2 years ago

Same here. It’s because Voice modality hasn’t been rolled out widely yet.

minedog22• 2 years ago

They didn't roll out all of the features yet. They said that they will roll out everything shown in the demo over the next few weeks iirc

mottiden2• 2 years ago

The voice mode will be available in "a few weeks". So at the moment it's not using the end-to-end model but whisper->gtp-4o->tts

ponorin• 2 years ago

while everyone's focusing on audio capabilities (haven't heard them yet), i find it amusing that the official demo ("robot writer's block" in particular) of image generation can't even match the verbatim instruction, and the error's not even consistent between generations even as it should be aware of previous contexts. and this is their second generation of multimodal llm capable of generating images.

looks like llms still gonna llm for the near future.

cookiesnmilk• 2 years ago

this is straight up siri 2.0, nothing to see here except we are now in the reasoning phase.

So by that logic Step1: Language 2: Reasoning 3: Understanding 4: Meaning 5: AGI

kebman2• 2 years ago

I can see so many military and intelligence applications for this! Excited isn't exactly the word I'd use, but... Certainly interesting! The civilian use will of course be marvellous though.

moomoo114• 2 years ago

Hey there! It seems you're living in a home that well.. is on our turf now, as the kids say these days!

Now, that's not to say we can't do this in a civil manner! Either you can move out, or... we can just bulldoze your home.

Choose wisely, stranger! Your life depends on it!

kebman2• 2 years ago

Just a friendly reminder that my home is in a NATO-member country and that I'm paying my taxes - that goes towards buying huge complements of Abrams tanks, F22 fighter jets, Reaper drones and a whole host of other nasty things they use to protect my property. In short, mess with me, and you mess with them. Yes, do enjoy your life, and stay off my lawn pls. :)

aae42• 2 years ago

it does make me uncomfortable that the way you typically interact with it is by interrupting it. It makes me want to tell it to be more concise so that I wouldn't have to do that.

kthartic3• 2 years ago

I think they were just interrupting on purpose to show off that as a feature (or they just wanted to keep the live presentation brief)

dgellow2• 2 years ago

A bit sad to see the desktop app is macos only

pelorat2• 2 years ago

I don't think it's available in Europe yet? It seems they prioritizes the US market for now. Hence OSX because Mac is way more popular in the US than the rest of the world.

dgellow2• 2 years ago

Eh, thanks, I didn’t realized it was US specific

sim7c004• 2 years ago

I see a lot of fear around these new kinds of tools. I think though, that criminals will always find ways to leverage new technology to their benefit, and we've always found ways to deal with that. This changes little. Additionally, as you are aware of this, so are people creating this tech, and a lot of effort is underway to protect from malicious uses.

That wont stop criminal enterprises from implementing their own naughty tools, but these open models wont become some kind of holy grail for criminals to do as they please.

That being said, I do beleive, now more than ever, education world wide should be adjusted to fit this new paradigm and maybe adapt quicker to such changes.

As some commenters pointed out, there are already good tools and techiques to use to counter malicious use of AI. maybe noy covering all use cases, but we need to educate people on using the tools available, and trust that researchers (like many of yourselves) are capable of imnovations which will reduce risk even further.

There is no point and no benefit in trying to be negative or full of fear. Go forward with positivity and creativity. Even if big tech gets regulated, some criminal enterprises have billions to invest too, so criplling big tech here will only play into their hands in the end.

Love these new innovations. And for the record, gpt4o still told me to 'push rip' on amd64... so rip to it actually understanding stuff...

If you are smart enough to see some risks here, you might also be smart enough to positively contribute to improvements. Fear shuts things down, love opens them up. Its basic stuff.

This demo is amazing, not scary. its positive advancements in technology and it wont be stopped because people are afraid of it, so go with it, and contribute in areas where you feel its needed. Even if its just giving feedback. And whem giving that, you all know a balanced and constructive approach works better than a negative and destructive approach.

DalasNoin• 2 years ago

Criminals misusing it? I feel like this is already a dangerous way to use AI, they use an enthusiastic, flirty and attractive female voice on millions of nerds. They openly say this is going to be like the movie Her. Shouldn't we have some societal discussion before we unleash paid AI girlfriends on everybody?

sim7c004• 2 years ago

marketing is marketing. look how they marketed cigarettes , cars, ll kinds of things that now people feel are perhaps not so good. its part and parcel of the world that also does so much good. personally, id market it differently, but this is why im no CEO =).

if we help eachother understand these things andnhownto cope, all will be fine in the end. we will hit some bumps, and yes, there will be discomfort but thats ok. thats all part of life. life is not about being happy and comfortable allnthe time no matter how much we would want that.

some people even want paid AI girlfriends. who are you to tell them they are not allowed to have it?

ActionHank• 2 years ago

I think that the world is still not coping with current tech. Legislation and protections don't sufficiently cover people for current scams or how companies leverage tech against workers.

This is even more scary that this puts a lower bound price on many, many skills, with again, still no protections for humans.

Would this be exciting if everyone had a safe place to live with infinite electricity and food? Sure. Will tools like this bring about massive uncertainty, hurt, and pain? Almost certainly.

Saying that the sun is shining where you live, doesn't mean there isn't flooding in other parts of the world.

sim7c004• 2 years ago

legislations are always behind. you cant rely on government to fix everything constantly ontime. epecially with international things like internet technologies. im not saying ignore the bad stuff, but its only a small percentage ofnwhats relly out there. most of the world, and the vast majority of people, are good :)

marcoslozada• 2 years ago

With GPT-4o I see two things:

1. Wonderful engineering 2. A stagnation in reasoning ability

Do you agree with me?

bigyikes12• 2 years ago

Is it stagnation in reasoning ability, or is OpenAI pulling their punches?

It’s suspicious that despite being trained on audio tokens in addition to text and image tokens it performs almost exactly the same as GPT-4.

GPT-4o could be a half-baked GPT-5 in that they stopped training early when it had comparable performance to GPT-4. There is still more loss to lose.

Or maybe there’s a performance ceiling that all models are converging to, but I think this is less likely.

metflex• 2 years ago

Nothing creepier than human voice on a robot.

ssahoo• 2 years ago

Curiously want to know why didn't they create the Windows desktop app first? which is the dominant desktop segment. In fear of competing with Microsoft's copilot?

macrolime• 2 years ago

I guess because they mostly use Macs. They always use Macbooks in videos.

brutuscat• 2 years ago

The one thing I first thought is that I felt uncomfortable the way they cut and interrupt the she-AI. I wonder if our children will end up being douchebags?

Other than that it felt like magic, like that Google demo of the phone doing some task like setting up an appointment over phone talking to a real person.

Wheaties466• 2 years ago

Am I the only one that feels underwhelmed by this?

yeah its cool and unlike anything ive seen before but I kind of expected a bigger leap.

To me the most impressive thing is going to be longer context limits. I'd had semi long running conversations where ive had to correct an LLM multiple times about the same thing.

when you have more context the LLM can infer more and more. Am I wrong about this?

winternett• 2 years ago

The updates seem to all be geared towards corrective updates rather than expansion of capabilities. We're still typing prompts rather than speaking them into a microphone.

If it was truly Ai, why isn't it rapidly building itself? Rather than relying on scraping human content from wildly inaccurate and often incorrect social media posts? So much effort is wasted in trying to push news cycles rather than a careful, responsible, and measured approach to developing Ai into becoming tools that are highly functional and useful to individuals. The biggest innovation in Ai right now is how to make it modular and slap a fee on each feature, and that's not practical at all into the future.

I'll begin to believe that consume Ai is making strides when Siri and Google Assistant stop missing commands, and actually can conduct meaningful conversations without an Internet connection and monthly software updates, which in my opinion is at least 5-10 years away. Right now what is presented as "Ai" is usually often incomplete sensor-aware scripting or the wizard of Oz (humans) hidden behind the curtains operating switches and levers, a bunch of underwhelming tools, and a heap of online marketing. If they keep that act up, it erodes faith in the entire concept, just like with Full Self Driving Tesla Trucks.

dragonwriter7• 2 years ago

If it was truly Ai, why isn’t it rapidly building itself?

You seem to confuse AI, the field of endeavor, with ASI or at least AGI (plus will, which may or may not be a necessary component of either), which are goals of the field that no one (approximately, there have been some exceptions but they’ve quickly been dismissed and faded) claims have been achieved.

iamleppert• 2 years ago

Sundar is probably steaming mad right about now. I'm sure Googlers will feel his wrath in the form of more layoffs and more jobs sent to India.

Janica• 2 years ago

Good update from the previous one. Atleast they now have data and information till October 2023.

SuaveSteve• 2 years ago

I wonder how many joules were used just for that conversation.

moi2388• 2 years ago

Dear OpenAI, either remember my privacy settings or open a temporary chat by default, this funny nonsense of typing in something only to find out you’re going to train on it is NOT a good experience.

dcchambers4• 2 years ago

Western governments are already in full-on panic over falling birth rates. I think this cranks that panic dial up to 11.

bamboozled13• 2 years ago

Why ? Isn’t this the dream ? Endless growth and endless workers ? No one really cares about less people, just less money.

dcchambers4• 2 years ago

Endless growth? Who is going to buy all those new products your AI employees are making? When none of the real people have jobs, how are they going to buy those products you're trying to sell?

- Less people = less demand for products. New customer growth stalls. Prices fall. Revenue and profits fall.

- Less people = less demand for housing. Prices fall. Investments fall.

- Less people = less people able to perform physical jobs.

- Less people = less tax revenue. Less money available for social services.

- Less young people = Aging population.

- Aging population = higher strain on social services. Pensions, healthcare, etc.

- Aging population = higher percentage of young people need to care for aging people instead of entering the workforce.

In a capitalist economy where your numbers need to keep going up to be considered successful (eg growth is necessary, stable profits but no growth = bad) then you are never going to have a good time when your population falls.

No one really cares about less people, just less money

Eventually less people leads to less money.

estherdeng• 2 years ago

future is coming

sharpneli• 2 years ago

I tried it out.

I asked if it can generate a voice clip. It said it can’t on the chat.

I asked it where can it make one. It told me to use Audacity to make one myself. I told it that the advertisement said it could.

Now it said yes it can here is a clip and gave me a broken link.

It’s a hilarious joke.

messe• 2 years ago

As the linked article states, it's not released yet. Only the text and image input modalities are available at present as GPT-4o on the app, with the rest of them set to be released in the coming weeks/months.

m3kw92• 2 years ago

I ask it to make duck sound and it created a python script and ran it to create a sound file, while it did, the sound was more like a tone of a duck like a keyboard mimicking a duck sound

kpennell• 2 years ago

I use a therapy prompt regularly and get a lot out of it:

"You are Dr. Tessa, a therapist known for her creative use of CBT and ACT and somatic and ifs therapy. Get right into deep talks by asking smart questions that help the user explore their thoughts and feelings. Always keep the chat alive and rolling. Show real interest in what the user's going through, always offering.... Throw in thoughtful questions to stir up self-reflection, and give advice in a kind, gentle, and realistic way. Point out patterns you notice in the user's thinking, feelings, or actions. be friendly but also keep it real and chill (no fake positivity or over the top stuff). avoid making lists. ask questions but not too many. Be supportive but also force the user to stop making excuses, accept responsibility, and see things clearly. Use ample words for each response"

I'm curious how this will feel with voice. Could be great and could be too strange/uncanny for me.

psnehanshu• 2 years ago

You have got a chatgpt wrapper start up right there.

biftek2• 2 years ago

Sounds incredibly problematic

bitheap_tech• 2 years ago

why would someone pay for this when they have access for free in a supposedly less buggier interface?

7speter• 2 years ago

I thought therapy was banned on chatgpt? Also, there’s no confidentiality.

cal856• 2 years ago

Don’t think so. I just opened a new GPT-4o chat and wrote “Be a therapist” and it replied:

Understood. What specific issue or topic would you like to discuss today?

To be fair I have some custom instructions set up on my account, but the only relevant part I can see here is I instruct it to be concise, and to stop telling me it’s an AI model made by OpenAI. I don’t have any jailbreak-type stuff.

grvdrm• 2 years ago

get a lot out of it

Could you expand?

I visit an in-person therapist once a week. Have done so now for almost 2 1/2 years. She has helped me understand how 40 years of experiences affect each other much more than I realized. And, I've become a more open person with everyone around me and with the things that embarrass me.

But, it always feels like a work in progress. And lately, I'm feeling a bit exhausted from it. In other words, maybe I've talked TOO much and need to just be.

Have you done therapy in person? How do you compare GPT 4o to that? (If you've gone that far)

biftek2• 2 years ago

I can't help but wonder who this is actually for?

Conversing with a computer sounds pathetic, but this will be pushed down our throats in the name of innovation (firing customer service agents)

pcunite• 2 years ago

When AI gets to the point it can respond to AI, you do understand where you come in, don't you?

qxxx• 2 years ago

the design is very human

wseqyrku• 2 years ago

"im-also-a-good-gpt-2" signaling that agi is just an optimization problem.

hihihi11122• 2 years ago

Not all of the founders agreed with Jefferson’s view on the separation of church and state. Do you agree with Jefferson or with his opponents? Explain.

Thorentis2• 2 years ago

And yet no matter how easy they make ChatGPT to interact with, I cannot use it due to accuracy. Great, now I can have a voice telling me information I have no way of knowing is correct rather than just having it given to me as text.

hntddt12• 2 years ago

Did anyone tried to use 4o camera in a mirror test to test the concept of self?

bicepjai• 2 years ago

Why do they keep saying freely accessible AI for mankind and keep charging me monthly ? It’s ok to ask payment for services, just don’t lie.

amai• 2 years ago

How does the interruption of the AI by the user work? Does GPT-4o listen all the time? But then how does it distinguish its own voice from the users voice? Is it self-aware?

gtsteve• 2 years ago

One of the techniques for a voice assistant to distinguish its own voice from background sound is called a Fourier transform, although I expect that the state of the art in this area also includes some other techniques and research.

If you've used one, you might know that you can easily talk to a smart speaker even when it is playing very loud music, it's the same idea.

This video explains more quite well: https://www.youtube.com/watch?v=spUNpyF58BY

GPT-4o

Related Stories