GPT-4.5

https://openai.com/index/introducing-gpt-4-5/

1136 971 1 year ago

throwup2382• 1 year ago

At this point I think the ultimate benchmark for any new LLM is whether or not it can come up with a coherent naming scheme for itself. Call it “self awareness.”

lenerdenator• 1 year ago

The people naming them really took the "just give the variable any old name, it doesn't matter" advice from Programming 101 to heart.

sho_hn2• 1 year ago

This is why my new LLM portfolio is Foo, Bar and Baz.

nopelynopington2• 1 year ago

3,3.5,4,4o,4.5

I had my money on 4oz

camwhite• 1 year ago

I can't wait for fireship.io and the comment section here to tell me what to think about this

mjburgess• 1 year ago

You appear to have the direction of causation reversed.

(In that fireship does the same)

lysace• 1 year ago

I wonder if fireship reaction video scripts to AI models based on HN comments can be automated using said AI models.

rob• 1 year ago

I bet simonw will be adding it to `llm` and someone will be pasting his highlights here right after. Until then, my mind will remain a blank canvas.

whisper_yb• 1 year ago

wowsers!

bhouston4• 1 year ago

A bit better at coding than ChatGPT 4o but not better than o3-mini - there is a chart near the bottom of the page that is easy to overlook:

- ChatGPT 4.5 on AWS Bench verified: 38.0%

- ChatGPT 4o on AWS Bench verified: 30.7%

- OpenAI o3-mini on AWS Bench verified: 61.0%

BTW Anthropic Claude 3.7 is better than o3-mini at coding at around 62-70% [1]. This means that I'll stick with Claude 3.7 for the time being for my open source alternative to Claude-code: https://github.com/drivecore/mycoder

[1] https://aws.amazon.com/blogs/aws/anthropics-claude-3-7-sonne...

logicchains• 1 year ago

BTW Anthropic Claude 3.7 is better than o3-mini at coding at around 62-70% [1]. This means that I'll stick with Claude 3.7 for the time being for my open source alternative to Claude-code

That's not a fair comparison as o3-mini is significantly cheaper. It's fine if your employer is paying, but on a personal project the cost of using Claude through the API is really noticeable.

cheema33• 1 year ago

That's not a fair comparison as o3-mini is significantly cheaper. It's fine if your employer is paying...

I use it via Cursor editor's built-in support for Claude 3.7. That caps the monthly expense to $20. There probably is a limit in Claude for these queries. But I haven't run into it yet. And I am a heavy user.

QuadmasterXLII• 1 year ago

If open ai offers a more expensive model (4.5) and a cheaper model (3 mini) and both are worse, it starts to be a fair comparison

ehsanu1• 1 year ago

It's the other way around on their new SWE-Lancer benchmark, which is pretty interesting: GPT-4.5 scores 32.6%, while o3-mini scores 10.8%.

Topfi4• 1 year ago

To put that in context, Claude 3.5 Sonnet (new), a model we have had for months now and which from all accounts seems to have been cheaper to train and is cheaper to use, is still ahead of GPT-4.5 at 36.1% vs 32.6% in SWE-Lancer Diamond [0]. The more I look into this release, the more confused I get.

[0] https://arxiv.org/pdf/2502.12115

_cs2017_• 1 year ago

I don't see Claude 3.7 on the official leaderboard. The top performer on the leaderboard right now is o1 with a scaffold (W&B Programmer O1 crosscheck5) at 64.6%: https://www.swebench.com/#verified.

If Claude 3.7 achieves 70.3%, it's quite impressive, it's not far from 71.7% claimed by o3, at (presumably) much, much lower costs.

aoeusnth13• 1 year ago

I doubt o3s costs will be lower for that performance. They juice their benchmark results by letting it spend $100k in thinking tokens.

pawelduda• 1 year ago

Does the benchmark reflect your opinion on 3.7? I've been using 3.7 via Cursor and it's noticeably worse than 3.5. I've heard using the standalone model works fine, didn't get a chance to try it yet though.

jasonjmcghee2• 1 year ago

personal anecdote - claude code is the best llm devx i've had.

bingdig2• 1 year ago

Not seeing it available in the app or on ChatGPT.com with a pro subscription.

hidelooktropic3• 1 year ago

It’s not supposed to be yet

bingdig2• 1 year ago

"Available to Pro users and developers worldwide" "Try in ChatGPT"

SamvitJ• 1 year ago

One comparison I found interesting... I think GPT-4o has a more balanced answer!

What are your thoughts on space exploration?

GPT-4.5:

Space exploration isn't just valuable—it's essential.

People often frame it as a luxury we pursue after solving Earth-bound problems. But space exploration actually helps us address those very challenges: climate change (via satellite monitoring), resource scarcity (through asteroid mining), and existential risk (by enabling eventual multi-planetary living). It also drives technological innovation, economic growth, and inspires humanity toward collective achievement.

We don't explore space to escape Earth. We explore space to better understand, protect, and sustain life on Earth.

What are your thoughts on space exploration?

GPT-4o:

Space exploration is one of the most ambitious and profound endeavors humanity has undertaken. It expands our understanding of the universe, drives technological innovation, and inspires generations to think beyond the boundaries of Earth.

From a scientific perspective, space exploration helps us address fundamental questions: Are we alone in the universe? How did the cosmos begin? Can we colonize other planets? Missions like the James Webb Space Telescope, Mars rovers, and deep-space probes continuously reshape our understanding of physics, planetary science, and even biology.

From a practical standpoint, space exploration fuels advancements in materials science, AI, robotics, and medicine. Many everyday technologies—GPS, satellite communications, weather forecasting, and even medical imaging—stem from space research.

There's also the existential argument: humanity's long-term survival may depend on our ability to become a multi-planetary species. While colonizing Mars or mining asteroids is far from easy, these efforts could help us safeguard civilization against existential threats like climate change or asteroid impacts.

That said, space exploration also raises ethical and economic questions. Should we focus more on Earth’s problems before investing trillions in space? How do we prevent the same exploitative behaviors that shaped colonial history from repeating on other planets?

Overall, I see space exploration as a necessary pursuit—not at the cost of solving problems on Earth, but as a way to advance our knowledge, drive innovation, and secure a future beyond our home planet. What’s your take?

gh0stcat• 1 year ago

Yeah, I also found it odd that they seem to be implying that an incredibly biased answer (as in 4.5) is better. In general, I find the tone more polarizing and not exactly warm as they advertised in the release video.

basisword• 1 year ago

As a benchmark, why do you find the 'opinion' of an LLM useful? The question is completely subjective. Edit: Genuinely asking. I'm assuming there's a reason this is an important measure.

Topfi4• 1 year ago

Not OP, but likely because that was the only metric/benchmark/however you want to call it OpenAI showcased in the stream and on the blog to highlight the improvement between 4o and 4.5. To say that this is not really a good metric for comparison, not least because prompting can have a massive impact in this regard, would be an understatement.

Chamix2• 1 year ago

Indeed, and the difference could in essence be achieved yourself with a different system prompt on 4o. What exactly is 4.5 contributing here in terms of a more nuanced intelligence?

The new RLHF direction (heavily amplified through scaling synthetic training tokens) seems to clobber any minor gains the improved base internet prediction gains might've added.

nprateem• 1 year ago

"X isn't just Y - it's Z. [Waffle]. By doing X, you can YY. Remember, ZZ. [Final superfluous sentence]"

God I hate reading what crapgpt writes.

ThouYS• 1 year ago

hmm.. not really the direction I expected them to go

doctoboggan3• 1 year ago

I am beginning to think these human eval tests are a waste of time at best, and negative value at worst. Maybe I am being snobby, but I don't think the average human is able to properly evaluate usefulness, truthfulness, or other metrics that I actually care about. I am sure this is good for openAI since if more people like what the hear, they are more likely come back.

I don't want my AI more obsequious, I want it more correct and capable.

My only use case is coding though, so maybe I am not representative of their usual customers?

dboreham2• 1 year ago

The SuperTuring era.

onlyrealcuzzo3• 1 year ago

I want it more correct and capable.

How is it supposed to be more correct and capable if these human eval tests are a waste of time?

Once you ask it to do more than add two numbers together, it gets a lot more difficult and subjective to determine whether it's correct and how correct.

doctoboggan3• 1 year ago

I agree it's a hard problem. I think there are a number of tests out there however that are able to objectively test capability and truthfulness.

I've read reports that some of the changes that are preferred by human evaluators actually hurt the performance on the more objective tests.

bloomingkales2• 1 year ago

These eval tests are just an anchor point to measure distance from, but it's true, picking the anchor point is important. We don't want to measure in the wrong direction.

ekojs• 1 year ago

Because of this, we’re evaluating whether to continue serving it in the API long-term as we balance supporting current capabilities with building future models.

Seems like it's not going to be deployed for long.

$75.00 / 1M tokens for input

$150.00 / 1M tokens for output

That's crazy prices.

bguberfain• 1 year ago

Until GPT-4.5, GPT-4 32K was certainly the most heavy model available at OpenAI. I can imagine the dilemma between to keep it running or stop it to free GPU for training new models. This time, OpenAI was clear whether to continue serving it in the API long-term.

jsheard2• 1 year ago

or stop it to free GPU for training new models.

Don't they use different hardware for inference and training? AIUI the former is usually done on cheaper GDDR cards and the latter is done on expensive HBM cards.

Chamix2• 1 year ago

It's interesting to compare the cost of that original gpt-4 32k(0314) vs gpt-4.5:

$60/M input tokens vs $75/M input tokens

$120/M output tokens vs $150/M output tokens

daemonologist• 1 year ago

Imagine if they built a reasoning model with costs like these. Sometimes it seems like they're on a trajectory to create a model which is strictly more capable than I am but which costs 100x my salary to run.

jes5199• 1 year ago

if you still get a moore’s law halving every couple years, it becomes competitive in, uh, about thirteen years?

nickreese• 1 year ago

Is it just me or is having the AI help you self sensor (as shown in the demo live stream: https://www.youtube.com/watch?v=cfRYp0nItZ8)... pretty dystopian?

nopelynopington2• 1 year ago

Oh this makes sense. chatGPT results have taken a nose dive in quality lately.

It couldn't write a simple rename function for me yesterday, still buggy after seven attempts.

I'm more and more convinced that they dumb down the core product when they plan to release a new version to make the difference seem bigger.

wilg2• 1 year ago

99% chance that's confirmation bias

nomel3• 1 year ago

Sam tweeted that they're running out of computer. I think it's reasonable to think they may serve somewhat quantized models when out of capacity. It would be a rational business decision that would minimally disrupt lower tier ChatGPT users.

Anecdotally, I've noticed what appears to be drops in quality, some days. When the quality drops, it responds in odd ways when asked what model it is.

logicallee2• 1 year ago

It couldn't write a simple rename function for me yesterday, still buggy after seven attempts.

I'm surprised and a bit nervous about that. We intend to bootstrap a large project with it!!

Both ChatGPT 4o (fast) and ChatGPT o1 (a bit slower, deeper thinking) should easily be able to do this without fail.

Where did it go wrong? Could you please link to your chat?

About my project: I run the sovereign State of Utopia (will be at stateofutopia.com and stofut.com for short) which is a country based on the idea of state-owned, autonomous AI's that do all the work and give out free money, goods, and services to all citizens/beneficiaries. We've built a chess app (i.e. a free source of entertainment) as a proof of concept though the founder had to be in the loop to fix some bugs:

https://taonexus.com/chess.html

and a version that shows obvious blunders, by showing which squares are under attack:

https://taonexus.com/blunderfreechess.html

One of the largest and most complicated applications anyone can run is a web browser. We don't have a web browser built, but we do have a buggy minimal version of it that can load and minimally display some web pages, and post successsfully:

https://taonexus.com/publicfiles/feb2025/84toy-toy-browser-w...

It's about 1700 lines of code and at this point runs into the limitations of all the major engines. But it does run, can load some web pages and can post successfully.

I'm shocked and surprised ChatGPT failed to get a rename function to work, in 7 attempts.

UrineSqueegee2• 1 year ago

with 4.5? Why? It's only meant for creative writing.

anti-soyboy4• 1 year ago

Yep I realized about that many time ago, they are literally scammers

skepticATX• 1 year ago

How many still believe that scaling up base models will lead to AGI?

tiahura2• 1 year ago

Dan Ives

oblio4• 1 year ago

The guy is a finance major...

Mizza3• 1 year ago

Sounds like it's a distill of O1? After R1, I don't care that much about non-reasoning models anymore. They don't even seem excited about it on the livestream.

I want tiny, fast and cheap non-reasoning models I can use in APIs and I want ultra smart reasoning models that I can query a few times a day as an end user (I don't mind if it takes a few minutes while I refill a coffee).

Oh, and I want that advanced voice mode that's good enough at transcription to serve as a babelfish!

After that, I guess it's pretty much all solved until the robots start appearing in public.

sebastiennight16• 1 year ago

Probably not a distill of o1, since o1 is a reasoning model and GPT4.5 is not. Also, OpenAI has been claiming that this is a very large model (and it's 2.5x more expensive than even OG GPT-4) so we can assume it's the biggest model they've trained so far.

They'll probably distill this one into GPT-4.5-mini or such, and have something faster and cheaper available soon.

Mizza3• 1 year ago

There are plenty of distills of reasoning models now, and they said in they livestream they used training data from "smaller models" - which is probably every model ever considering how expensive this one is.

eightysixfour10• 1 year ago

It isn't even vaguely a distill of o1. The reasoning models are, from what we can tell, relatively small. This model is massive and they probably scaled the parameter count to improve factual knowledge retention.

They also mentioned developing some new techniques for training small models and then incorporating those into the larger model (probably to help scale across datacenters), so I wonder if they are doing a bit of what people think MoE is, but isn't. Pre-train a smaller model, focus it on specific domains, then use that to provide synthetic data for training the larger model on that domain.

Mizza3• 1 year ago

You can 'distill' with data from a smaller, better model into a larger, shittier one. It doesn't matter. This is what they said they did on the livestream.

valine• 1 year ago

My impression is that it’s a massive increase in the parameter count. This is likely the spiritual successor to GPT4 and would have been called GPT5 if not for the lackluster performance. The speculation is that there simply isn’t enough data on the internet to support yet another 10x jump in parameters.

O1-mini is a distill of O1. This definitely isn’t the same thing.

rvz• 1 year ago

You should really be paying attention to what DeepSeek AI open sources next.

This announcement by OpenAI was already expected: [0]

[0] https://x.com/sama/status/1889755723078443244

brokensegue• 1 year ago

API price is crazy high. This model must be huge. Not sure this is practical

jdprgm3• 1 year ago

Wow you aren't kidding, 30x input price and 15x output price vs 4o is insane. The pricing on all AI API stuff changes so rapidly and is often so extreme between models it is all hard to keep track of and try to make value decisions. I would consider a 2x or 3x price increase quite significant, 30x is wild. I wonder how that even translates... there is no way the model size is 30 times larger right?

jdprgm3• 1 year ago

Per Altman on X: "we will add tens of thousands of GPUs next week and roll it out to the plus tier then". Meanwhile a month after launch rtx 5000 series is completely unavailable and hardly any restocks and the "launch" consisted of microcenters getting literally tens of cards. Nvidia really has basically abandoned consumers.

apsec1123• 1 year ago

AI GPUs are bottlenecked mostly by high-bandwidth memory (HBM) chips and CoWoS (packaging tech used to integrate HBM with the GPU die), which are in short supply and aren't found in consumer cards at all

jiggawatts• 1 year ago

You would think that by now they would have done something to ramp production capacity…

bhouston4• 1 year ago

Altman's claim and NVIDIA's consumer launch supply problems may be related - OpenAI may be eating up the GPU supply...

_zoltan_• 1 year ago

OpenAI is not purchasing consumer 5090s... :)

sunaookami• 1 year ago

This seems very rushed because of DeepSeek's R1 and Anthropic's Claude 3.7 Sonnet. Pretty underwhelming, they didn't even show programming? In the livestream, they struggled to come up with reasons why I should prefer GPT-4.5 over GPT-4o or o1.

apsec1123• 1 year ago

At least according to WSJ, they had planned to release it earlier but struggled to get the model quality up, especially relative to cost

bhouston4• 1 year ago

they do have coding benchmarks, I summarized them here: https://news.ycombinator.com/item?id=43197955

bitshiftfaced• 1 year ago

This strikes me as the opposite of rushed. I get the impression that they've been sitting on this for a while and couldn't make it look as good as previous improvements. At some point they had to say, "welp here it is, now we can check that box and move on."

Topfi4• 1 year ago

Considering both this blog post and the livestream demos, I am underwhelmed. Having just finished the stream, I had a real "was that all" moment, which on one hand shows how spoiled I've gotten by new models impressing me, but on another feels like OpenAI really struggles to stay ahead of their competitors.

What has been shown feels like it could be achieved using a custom system prompt on older versions of OpenAIs models, and I struggle to see anything here that truly required ground-up training on such a massive scale. Hearing that they were forced to spread their training across multiple data centers simultaneously, coupled with their recent release of SWE-Lancer [0] which showed Anthropic (Claude 3.5 Sonnet (new) to be exact) handily beating them, I was really expecting something more than "slightly more casual/shorter output", which again, I fail to see how that wasn't possible by prompting GPT-4o.

Looking at pricing [1], I am frankly astonished.

Input: $75.00 / 1M tokens Cached input: $37.50 / 1M tokens Output: $150.00 / 1M tokens

How could they justify that asking price? And, if they have some amazing capabilities that make a 30-fold pricing increase justifiable, why not show it? Like, OpenAI are many things, but I always felt they understood price vs performance incredibly well, from the start with gpt-3.5-turbo up to now with o3-mini, so this really baffles me. If GPT-4.5 can justify such immense cost in certain tasks, why hide that and if not, why release this at all?

[0] https://github.com/openai/SWELancer-Benchmark

[1] https://openai.com/api/pricing/

Bjorkbat• 1 year ago

My first thought seeing this and looking at benchmarks was that if it wasn’t for reasoning, then either pundits would be saying we’ve hit a plateau, or at the very least OpenAI is clearly in 2nd place to Anthropic in model performance.

Of course we don’t live in such a world, but I thought of this nonetheless because for all the connotations that come with a 4.5 moniker this is kind of underwhelming.

uh_uh• 1 year ago

Pundits were saying that deep learning has hit a plateau even before the LLM boom.

mvdtnz2• 1 year ago

How could they justify that asking price?

They're still selling $1 for <$1. Like personal food delivery before it, consumers will eventually need to wake up to this fact - these things will get expensive, fast.

spiderfarmer2• 1 year ago

Let a thousand providers bloom.

Ekaros• 1 year ago

I generally question how wide spread willingness to pay for the most expensive product is. And will most users of those who actually want AI go with ad ridden lesser models...

phillipcarter• 1 year ago

I read this more as "we are releasing a model checkpoint that we didn't optimize yet because Anthropic cranked up the pressure"

josh-sematic• 1 year ago

One difference with food delivery/ride share: those can only have costs reduced so far. You can only pick up groceries and drive from A to B so quickly. And you can only push the wages down so far before you lose your gig workers. Whereas with these models we’ve consistently seen that a model inference that cost $1 several months ago can now be done with much less than $1 today. We don’t have any principled understanding of “we will never be able to make these models more efficient than X”, for any value of X that is in sight. Could the anticipated efficiencies fail to materialize? It’s possible but I personally wouldn’t put money on it.

BriggyDwiggs42• 1 year ago

I’ll probably stick to open models at that point.

sebzim45003• 1 year ago

This is often claimed on HN but there is no evidence that it is actually true.

sama has tweeted that they lose money on pro, but in general according to leaks chatgpt subscriptions are quite profitable. The reason the company isn't profitable in general is they spend billions on R&D.

tmaly2• 1 year ago

rethinking your comment "was that all" I am listening to the stream now and had a thought. Most of the new models that have come out in the past few weeks have been great at coding and logical reasoning. But 4o has been better at creative writing. I am wondering if 4.5 is going to be even better at creative writing than 4o.

maeil• 1 year ago

But 4o has been better at creative writing

In what way? I find the opposite, 4o's output has a very strong AI vibe, much moreso than competitors like Claude and Gemini. You can immediately tell, and instructing it to write differently (except for obvious caricatures like "Write like Gen Z") doesn't seem to help.

dingnuts4• 1 year ago

if you generate "creative" writing, please tell your audience that it is generated, before asking them to read it.

I do not understand what possible motivation there could be for generating "creative writing" unless you enjoy reading meaningless stories yourself, in which case, be my guest.

vjerancrnjak• 1 year ago

I still find all of them lacking on creative writing. The models are severely crippled by tokenization, complete lack of understanding of language rhythm.

They can’t generate a simple haiku consistently, something larger is more out of reach.

For example, give it a piece of poetry and ask for new verses and it just sucks at replicating the language structure and rhythm of original verses.

lasermike026• 1 year ago

I would rather pay for 4.5 by the query.

nycdatasci3• 1 year ago

Funny you should suggest that it seems like a revised system prompt: https://chatgpt.com/share/67c0fda8-a940-800f-bbdc-6674a8375f...

nycdatasci3• 1 year ago

In case there was any confusion, the referenced link shows 4.5 claiming to be “ChatGPT 4.0 Turbo”. I have tried multiple times and various approaches. This model is aware of 4.5 via search, but insists that it is 4 or 4 turbo. Something doesn’t add up. This cannot be part of the response to R1, Grok 3, and Claude 3.7. Satya’s decision to limit capex seems prescient.

swagmoney1606• 1 year ago

I have no idea how they justify $200/month for pro

energy1235• 1 year ago

The niche of GPT-4.5 is lower hallucations than any existing model. Whether that niche justifies the price tag for a subset of usecases remains to be seen.

energy1235• 1 year ago

Actually, this comment of mine was incorrect, or at least we don't have enough information to conclude this. The metric OpenAI are reporting is the total number of incorrect responses on SimpleQA (and they're being beaten by Claude Haiku on this metric...), which is a deceptive metric because it doesn't account for non-responses. A better metric would be the ratio of Incorrects to the total number of attempts.

petesergeant• 1 year ago

but on another feels like OpenAI really struggles to stay ahead of their competitors

on one hand. On the other hand, you can have 4o-mini and o3-mini back when you can pry them out of my cold dead hands. They're _fast_, they're _cheap_, and in 90% of cases where you're automating anything, they're all you need. Also they can handle significant volume.

I'm not sure that's going to save OpenAI, but their -mini models really are something special for the price/performance/accuracy.

anshumankmr4• 1 year ago

I suspect they may launch a GPT4.5Turbo with a price cut... GPT4/GPT432k etc were all pricier than the GPT4Turbo models which also came with the added context length.. but with this huge jump in price, even 4.5Turbo if it does come out would be pricier

zaptrem2• 1 year ago

GPT 4.5 pricing is insane: Price Input: $75.00 / 1M tokens Cached input: $37.50 / 1M tokens Output: $150.00 / 1M tokens

GPT 4o pricing for comparison: Price Input: $2.50 / 1M tokens Cached input: $1.25 / 1M tokens Output: $10.00 / 1M tokens

It sounds like it's so expensive and the difference in usefulness is so lacking(?) they're not even gonna keep serving it in the API for long:

GPT‑4.5 is a very large and compute-intensive model, making it more expensive than and not a replacement for GPT‑4o. Because of this, we’re evaluating whether to continue serving it in the API long-term as we balance supporting current capabilities with building future models. We look forward to learning more about its strengths, capabilities, and potential applications in real-world settings. If GPT‑4.5 delivers unique value for your use case, your feedback (opens in a new window) will play an important role in guiding our decision.

I'm still gonna give it a go, though.

MattSayar2• 1 year ago

Input price difference: 4.5 is 30x more

Output price difference:4.5 is 15x more

In their model evaluation scores in the appendix, 4.5 is, on average, 26% better. I don't understand the value here.

alwa• 1 year ago

If you ran the same query set 30x or 15x on the cheaper model (and compensated for all the extra tokens the reasoning model uses), would you be able to realize the same 26% quality gain in a machine-adjudicatible kind of way?

mirekrusin• 1 year ago

Einstein's IQ = 3.5x chimpanzees IQs, right?

minimaxir2• 1 year ago

Sam Altman's explanation for the restriction is a bit fluffier: https://x.com/sama/status/1895203654103351462

bad news: it is a giant, expensive model. we really wanted to launch it to plus and pro at the same time, but we've been growing a lot and are out of GPUs. we will add tens of thousands of GPUs next week and roll it out to the plus tier then. (hundreds of thousands coming soon, and i'm pretty sure y'all will use every one we can rack up.)

g-mork• 1 year ago

release blog post author: this is definitely a research preview

ceo: it's ready

the pricing is probably a mixture of dealing with GPU scarcity and intentionally discouraging actual users. I can't imagine the pressure they must be under to show they are releasing and staying ahead, but Altman's tweet makes it clear they aren't really ready to sell this to the general public yet.

chefandy6• 1 year ago

I’m not an expert or anything, but from my vantage point, each passing release makes Altman’s confidence look more aspirational than visionary, which is a really bad place to be with that kind of money tied up. My financial manager is pretty bullish on tech so I hope he is paying close attention to the way this market space is evolving. He’s good at his job, a nice guy, and surely wears much more expensive underwear than I do— I’d hate to see him lose a pair powering on his Bloomberg terminal in the morning one of these days.

rebolek2• 1 year ago

Bad news: Sam Altman runs the show.

sebastiennight16• 1 year ago

I think it's fairer to compare it to the original GPT-4 which might the equivalent in term of "size" (though we don't have actual numbers for either).

GPT-4: Input $30.00 / 1M tokens ; Output $60.00 / 1M tokens

So 4.5 is 2.5x more expensive.

I think they announced this as their last non-reasoning model, so it was maybe with the goal of stretching pre-training as far as they could, just to see what new capabilities would show up. We'll find out as the community gives it a whirl.

I'm a Tier 5 org and I have it available already in the API.

minimaxir2• 1 year ago

The marginal costs for running a GPT-4-class LLM are much lower nowadays due to significant software and hardware innovations since then, so costs/pricing are harder to compare.

jstummbillig3• 1 year ago

Why would that be fairer? We can assume they did incorporate all learnings and optimizations they made post gpt-4 launch, no?

spoaceman7777• 1 year ago

There are some numbers on one of their Blackwell or Hopper info pages that notes the ability of their hardware in hosting an unnamed GPT model that is 1.8T params. My assumption was that it referred to GPT-4

Sounds to me like GPT 4.5 likely requires a full Blackwell HGX cabinet or something, thus OpenAI's reference to needing to scale out their compute more (Supermicro only opened up their Blackwell racks for General Availability last month, and they're the prime vendor for water-cooled Blackwell cabinets right now, and have the ability to throw up a GPU mega-cluster in a few weeks, like they did for xAI/Grok)

OldGreenYodaGPT2• 1 year ago

2x that price for the 32k context via API at launch. So nearly the same price, but you get 4x the context

harlanlewis3• 1 year ago

The price really is eye watering. At a glance, my first impression is this is something like Llama 3.1 405B, where the primary value may be realized in generating high quality synthetic data for training rather than direct use.

I keep a little google spreadsheet with some charts to help visualize the landscape at a glance in terms of capability/price/throughput, bringing in the various index scores as they become available. Hope folks find it useful, feel free to copy and claim as your own.

https://docs.google.com/spreadsheets/d/1foc98Jtbi0-GUsNySddv...

bennyg• 1 year ago

This is an amazing spreadsheet - thank you for sharing!

isoprophlex• 1 year ago

Thats... incredibly thorough. Wow. Thanks for sharing this.

Philpax• 1 year ago

Holy shit, that's incredible. You should publicise this more! That's a fantastic resource.

adinb• 1 year ago

I cannot overstate how good your shared spreadsheet is. Thanks again!

jnd0• 1 year ago

Thank you so much for sharing this!

bglusman• 1 year ago

very impressive... also interested in your trip planner, it looks like invite only at the moment, but... would it be rude to ask for an invite?

gwyllimj• 1 year ago

That is an amazing resource. Thanks for sharing!

dumpsterdiver2• 1 year ago

Nice, thank you for that (upvoted in appreciation). Regarding the absence of o1-Pro from the analysis, is that just because there isn't enough public information available?

krwiseman• 1 year ago

Awesome spreadsheet. Would a 3D graph of fast, cheap & smart be possible?

rendist• 1 year ago

Amazing, thank you so much for sharing this.

senordevnyc• 1 year ago

Hey, just FYI, I pasted your url from the spreadsheet title into Safari on macOS and got an SSL warning. Unfortunately I clicked through and now it works, so not sure what the exact cause looked like.

sfink2• 1 year ago

feel free to copy and claim as your own.

That's a nice sentiment, but I'd encourage you to add a license or something. The basic "something" would be adding a canonical URL into the spreadsheet itself somewhere, along with a notification that users can do what they want other than removing that URL. (And the URL would be described as "the original source" or something, not a claim that the particular version/incarnation someone is looking at is the same as what is at that URL.)

The risk is that someone will accidentally introduce errors or unsupportable claims, and people with the modified spreadsheet won't know that it's not The spreadsheet and so will discount its accuracy or trustability. (If people are trying to deceive others into thinking it's the original, they'll remove the notice, but that's a different problem.) It would be a shame for people to lose faith in your work because of crap that other people do that you have no say in.

mwigdahl• 1 year ago

Wow, what awesome information! Thanks for sharing!

jwr2• 1 year ago

This is incredibly useful, thank you for sharing!

6gvONxR4sf7o• 1 year ago

Not just for training data, but for eval data. If you can spend a few grand on really good labels for benchmarking your attempts at making something feasible work, that’s also super handy.

swyx2• 1 year ago

https://docs.google.com/spreadsheets/d/1foc98Jtbi0-GUsNySddv...

how do you do the different size circles and colored sequences like that? this is god tier skills

taurath• 1 year ago

What gets me is the whole cost structure is based on practically free services due to all the investor money. They’re not pulling in significant revenue with this pricing relative to what it costs to train the models, so the cost may be completely different if they had to recoup those costs, right?

crovona• 1 year ago

Very useful

swatcoder• 1 year ago

We look forward to learning more about its strengths, capabilities, and potential applications in real-world settings. If GPT‑4.5 delivers unique value for your use case, your feedback (opens in a new window) will play an important role in guiding our decision.

"We don't really know what this is good for, but spent a lot of money and time making it and are under intense pressure to announce new things right now. If you can figure something out, we need you to help us."

Not a confident place for an org trying to sustain a $XXXB valuation.

tempaccount420• 1 year ago

"We don't really know what this is good for, but spent a lot of money and time making it and are under intense pressure to announce new things right now. If you can figure something out, we need you to help us."

Where is this quote from?

riwsky2• 1 year ago

Said the quiet part out loud! Or as we say these days, “transparently exposed the chain of thought tokens”.

EA-3167• 1 year ago

Maybe if they build a few more data centers, they'll be able to construct their machine god. Just a few more dedicated power plants, a lake or two, a few hundred billion more and they'll crack this thing wide open.

And maybe Tesla is going to deliver truly full self driving tech any day now.

And Star Citizen will prove to have been worth it along along, and Bitcoin will rain from the heavens.

It's very difficult to remain charitable when people seem to always be chasing the new iteration of the same old thing, and we're expected to come along for the ride.

jodrellblank• 1 year ago

"Early testing shows that interacting with GPT‑4.5 feels more natural. Its broader knowledge base, improved ability to follow user intent, and greater “EQ” make it useful for tasks like improving writing, programming, and solving practical problems. We also expect it to hallucinate less."

"Early testing doesn't show that it hallucinates less, but we expect that putting that sentence nearby will lead you to draw a connection there yourself".

crazygringo2• 1 year ago

We don't really know what this is good for

Oh come on. Think how long of a gap there was between the first microcomputer and VisiCalc. Or between the start of the internet and social networking.

First of all, it's going to take us 10 years to figure out how to use LLM's to their full productive potential.

And second of all, it's going to take us collectively a long time to also figure out how much accuracy is necessary to pay for in which different applications. Putting out a higher-accuracy, higher-cost model for the market to try is an important part of figuring that out.

With new disruptive technologies, companies aren't supposed to be able to look into a crystal ball and see the future. They're supposed to try new things and see what the market finds useful.

fsndz3• 1 year ago

it's so over, pretraining is ngmi. maybe sam Altman was wrong after all ? https://www.lycee.ai/blog/why-sam-altman-is-wrong

amarcheschi2• 1 year ago

I have a professor who founded a few companies, one of these was funded by gates after he managed to spoke with him and convinced him to give him money. This guy is goat, and he always tells us that we need to find solutions to problems, not to find problems to our solutions. It seems at openai they didn't get the memo this time

pinkmuffinere3• 1 year ago

This is a very harsh take. Another interpretation is “We know this is much more expensive, but it’s possible that some customers do value the improved performance enough to justify the additional cost. If we find that nobody wants that, we’ll shut it down, so please let us know if you value this option”.

crystal_revenge2• 1 year ago

"We don't really know what this is good for, but spent a lot of money and time making it and are under intense pressure to announce new things right now. If you can figure something out, we need you to help us."

Having worked at my fair share of big tech companies (while preferring to stay in smaller startups), in so many of these tech announcement I can feel the pressure the PM had from leadership, and hear the quiet cries of the one to two experience engineers on the team arguing sprint after sprint that "this doesn't make sense!"

ummonk• 1 year ago

Conspiracy theory: they’re trying to tank the valuation so that Altman can buy it out at bargain price.

spaceman_2020• 1 year ago

Really don’t understand what’s the use case for this. The o series models are better and cheaper. Sonnet 3.7 smokes it on coding. Deepseek R1 is free and does a better job than any of OAI’s free models

roarcher3• 1 year ago

AI in general is increasingly a solution in search of a problem, so this seems about right.

0xDEAFBEAD• 1 year ago

There's a decent chance this model was originally called GPT-5, as well.

xnx• 1 year ago

ChatGPT has been coasting on name recognition since 4.

NewUser76312• 1 year ago

Damn this never worked for me as a startup founder lol. Need that Altman "rizz" or what have you.

jcgrillo3• 1 year ago

The fact they're raising prices so steeply is telling. This smells like desperation.

serjester• 1 year ago

I suppose this was their final hurrah after two failed attempts at training GPT-5 with the traditional pre-training paradigm. Just confirms reasoning models are the only way forward.

newfocogi• 1 year ago

I think this is the correct take. There are other axes to scale on AND I expect we'll see smaller and smaller models approach this level of pre-trained performance. But I believe massive pre-training gains have hit clearly diminished returns (until I see evidence otherwise).

granzymes• 1 year ago

Compared to OpenAI o1 and OpenAI o3‑mini, GPT‑4.5 is a more general-purpose, innately smarter model. We believe reasoning will be a core capability of future models, and that the two approaches to scaling—pre-training and reasoning—will complement each other. As models like GPT‑4.5 become smarter and more knowledgeable through pre-training, they will serve as an even stronger foundation for reasoning and tool-using agents.

jstummbillig3• 1 year ago

What it confirms, I think, is, that we are going to need a lot more chips.

usaar333• 1 year ago

For OpenAI perhaps? Sonnet 3.7 without extended thinking is quite strong. Swe-bench scores tie o3

DebtDeflation2• 1 year ago

GPT 5 is likely just going to be a router model that decides whether to send the prompt to 4o, 4o mini, 4.5, o3, or o3 mini.

crystal_revenge2• 1 year ago

Just confirms reasoning models are the only way forward.

Reasoning models are roughly the equivalent to allow Hamiltonian Monte-Carlo models to "warm up" (i.e. start sampling from the typical set). This, unsurprisingly, yields better results (after all LLMs are just fancy Monte-carlo models in the end). However, it is extremely unlikely this improvement is without pretty reasonable limitations. Letting your HMC warm up is essential to good sampling, but letting "warm up more" doesn't result in radically better sampling.

While there have been impressive results in efficiency of sampling from the typical set seen in LLMs these days, we're clearly not making the major improvements in the capabilities of these models.

techorange• 1 year ago

I wonder how much money they’re losing on it too even at those prices.

nialv73• 1 year ago

Looks like more signal that the scaling "law" is indeed faltering.

ur-whale2• 1 year ago

AI as it stands in 2025 is an amazing technology, but it is not a product at all.

As a result, OpenAI simply does not have a business model, even if they are trying to convince the world that they do.

My bet is that they're currently burning through other people's capital at an amazing rate, but that they are light-years from profitability

They are also being chased by fierce competition and OpenSource which is very close behind. There simply is no moat.

It will not end well for investors who sunk money in these large AI startups (unless of course they manage to find a Softbank-style mark to sell the whole thing to), but everyone will benefit from the progress AI will have made during the bubble.

So, in the end, OpenAI will have, albeit very unwillingly, fulfilled their original charter of improving humanity's lot.

jsheard2• 1 year ago

My bet is that they're currently burning through other people's capital at an amazing rate, but that they are light-years from profitability

The Information leaked their internal projections a few months ago, and apparently their own estimates have them losing $44B between then and 2029 when they expect to finally turn a profit, maybe.

whiplash4512• 1 year ago

Except that if OpenAI goes bust, very little of what they did will actually be released to human kind.

So their contribution was really to fuel a race for opensource (which they contributed little to). Pretty complex of an argument.

emptysongglass• 1 year ago

I've been a Plus user for a long time now. My opinion is there is very much a ChatGPT suite of products that come together to make for a mostly delightful experience.

Three things I use all the time:

- Canvas for proofing and editing my article drafts before publishing. This has replaced an actual human editor for me.

- Voice for all sorts of things, mostly for thinking out loud about problems or a quick question about pop culture, what something means in another language, etc. The Sol voice is so approachable for me.

- GPTs I can use for things like D&D adventure summaries I need in a certain style every time without any manual prompting.

vineyardmike• 1 year ago

As a result, OpenAI simply does not have a business model, even if they are trying to convince the world that they do.

They have a super popular subscription service. If they keep iterating on the product enough, they can lag on the models. The business is the product not the models and not the API. Subscriptions are pretty sticky when you start getting your data entrenched in it. I keep my ChatGPT subscription because it’s the best app on Mac and already started to “learn me” through the memory and tasks feature.

Their app experience is easily the best out of their competitors (grok, Claude, etc). Which is a clear sign they know that it’s the product to sell. Things like DeepResearch and related are the way they’ll make it a sustainable business - add value-on-top experiences which drive the differentiation over commodities. Gemini is the only competitor that compares because it’s everywhere in Google surfaces. OpenAI’s pro tier will surely continue to get better, I think more LLM-enabled features will continue to be a differentiator. The biggest challenge will be continuing distribution and new features requiring interfacing with third parties to be more “agentic”.

Frankly, I think they have enough strength in product with their current models today that even if model training stalled it’d be a valuable business.

jcgrillo3• 1 year ago

https://podcasts.apple.com/us/podcast/better-offline/id17305...

nyarlathotep_3• 1 year ago

AI as it stands in 2025 is an amazing technology, but it is not a product at all.

Here I'm assuming "AI" to mean what's broadly called Generative AI (LLMs, photo, video generation)

I genuinely am struggling to see what the product is too.

The code assistant use cases are really impressive across the board (and I'm someone who was vocally against them less than a year ago), and I pay for Github CoPilot (for now) but I can't think of any offering otherwise to dispute your claim.

It seems like companies are desperate to find a market fit, and shoving the words "agentic" everywhere doesn't inspire confidence.

Here's the thing: I remember people lining up around the block for iPhone releases, XBox launches, hell even Grand Theft Auto midnight releases.

Is there a market of people clamoring to use/get anything GenAI related?

If any/all LLM services went down tonight, what's the impact? Kids do their own homework?

JavaScript programmers have to remember how to write React components?

Compare that with Google Maps disappearing, or similar.

LLMs are in a position where they're forced onto people and most frankly aren't that interested. Did anyone ASK for Microsoft throwing some Copilot things all over their operating system? Does anyone want Apple Intelligence, really?

beefnugs2• 1 year ago

Yes: the real truth is, if there really was a good AI created, then we wouldnt even know about it existing until a billion dollar company takes over some industry with only a handful of developers in the entire company. Only then would hints spill out into the world that its possible.

No "good" AI will ever be open to everyone and relatively cheap, this is the same phenomenon as "how to get rich" books

yard2010• 1 year ago

Sir they are selling text by the ounce just like farmers sold tomatoes before Walmart, How is that not a business model?

jdprgm3• 1 year ago

If it really costs them 30x more surely they must plan on putting pretty significant usage limits on any rollout to the Plus tier and if that is the case i'm not sure what the point is considering it seems primarily a replacement/upgrade for 4o.

The cognitive overhead of choosing between what will be 6 different models now on chatGPT and trying to map whether a query is "worth" using a certain model and worrying about hitting usage limits is getting kind of out of control.

JohnnyMarcone3• 1 year ago

To be fair their roadmap states that gpt-5 will unify everything into one model in "months".

chollida12• 1 year ago

GPT 4.5 pricing is insane:
I'm still gonna give it a go, though.

Seems like the pricing is pretty rational then?

phito• 1 year ago

Not if people just try a few prompts then stop using it.

raytopia• 1 year ago

Now the real question about AI automation starts. Is it cheaper to pay a human to do the task or a AI company?

redox992• 1 year ago

It still not smart enough to replace for example customer service.

fragmede6• 1 year ago

Humans have all sorts of issues you have to deal with. Being hungover, not sleeping well, having a personality, being late to work, not being able to work 24/7, very limited ability to copy them. If there's a soulless generic office-droidGPT that companies could hire that would never talk back and would do all sorts of menial work without needing breaks or to use the bathroom, I don't know that we humans stand a chance!

I have a bunch of work that needs doing. I can do it myself, or I can hire one person to do it. I gotta train them and manage them and even after I train them theres still only going to be one of them, and it's subject to their availability. On the other hand, if I need to train an AI to do it, but I can copy that AI, and then spin them up/down like on demand computer in the cloud, and not feel remotely bad about spinning them down?

It's definitely not there yet, but it's not hard to see the business case for it.

rrrrrrrrrrrryan• 1 year ago

I was about to comment that humans consume orders of magnitude less energy, but then I checked the numbers, and it looks like an average person consumes way more energy throughout their day (food, transportation, electricity usage, etc) than GPT-4.5 would at 1 query per minute over 24 hours.

crooked-v• 1 year ago

Doubly so with how good Claude 3.7 Sonnet is at $3 / 1M tokens.

Hansenq• 1 year ago

GPT-4.5 is 15-30x more expensive than GPT-4o. Likely that much larger in terms of parameter count too. It’s massive!!

With more parameters comes more latent space to build a world model. No wonder its internal world model is so much better than previous SOTA

wavemode• 1 year ago

This has been my suspicion for a long time - OpenAI have indeed been working on "GPT5", but training and running it is proving so expensive (and its actual reasoning abilities only marginally stronger than GPT4) that there's just no market for it.

It points to an overall plateau being reached in the performance of the transformer architecture.

goatlover• 1 year ago

Certainly hope so. The tech billionaires are little to excited to achieve AGI and replace the workforce.

camdenreslink3• 1 year ago

That would certainly reduce my anxiety about the future of my chosen profession.

malthaus3• 1 year ago

but while there is a plateau in the transformer architecture, what you can do with those base models by further finetuning / modifying / enhancing them is still largely unexplored so i still predict mind-blowing enhancements yearly for this foreseeable future. if they validate openai's valuation and investment needs is a different question.

hintymad• 1 year ago

It sounds like it's so expensive and the difference in usefulness is so lacking(?) they're not even gonna keep serving it in the API for long

I guess the rationale behind this is paying for the marginal improvement. Maybe the next few percent of improvement is so important to a business that the business is willing to pay a hefty premium.

shawabawa32• 1 year ago

I wonder if the pricing is partly to discourage distillation, if they suspect r1 was distilled from gpt 4o

schneehertz• 1 year ago

Mainly to prevent you from using it

tomrod• 1 year ago

I can chew through 1MM tokens with a single standard (and optimized) call. This pricing is insane.

MangoCoffee• 1 year ago

one of the problem seem to be there's no alternative to Nvidia ecosystem. (the gpu + CUDA).

kridsdale32• 1 year ago

May I introduce you to Gemini 2.0

Fnoord• 1 year ago

ZLUDA can be used as compatibility glue, also you can use ROCm or even Vulcan with Ollama.

coliveira• 1 year ago

In other words, they want people to pay for the privilege of becoming beta testers....

wiremine• 1 year ago

GPT 4.5 pricing is insane: Price Input: $75.00 / 1M tokens Cached input: $37.50 / 1M tokens Output: $150.00 / 1M tokens
GPT 4o pricing for comparison: Price Input: $2.50 / 1M tokens Cached input: $1.25 / 1M tokens Output: $10.00 / 1M tokens

Their examples don't seem 30x better. :-)

ren_engineer• 1 year ago

hyperscalers in shambles, no clue why they even released this other than the fact they didn't want to admit they wasted an absurd amount of money for no reason

hn_throwaway_99• 1 year ago

The price is obviously 15-30x that of 4o, but I'd just posit that there are some use cases where it may make sense. It probably doesn't make sense for the "open-ended consumer facing chatbot" use case, but for other use cases that are fewer and higher value in nature, it could if it's abilities are considerably better than 4o.

For example, there are now a bunch of vendors that sell "respond to RFP" AI products. The number of RFPs that any sales organization responds to is probably no more than a couple a week, but it's a very time-consuming, laborious process. But the payoff is obviously very high if a response results in a closed sale. So here paying 30x for marginally better performance makes perfect sense.

I can think of a number of similar "high value, relatively low occurrence" use cases like this where the pricing may not be a big hindrance.

Manouchehri• 1 year ago

Yeah, agreed.

We’re one of those types of customers. We wrote an OpenAI API compatible gateway that automatically batches stuff for us, so we get 50% off for basically no extra dev work in our client applications.

I don’t care about speed, I care about getting the right answer. The cost is fine as long as the output generates us more profit.

janoc4• 1 year ago

And which use case will that make sense then for?

Esp. when they aren't even sure whether they will commit to offering this long term? Who would be insane enough to build a product on top of something that may not be there tomorrow?

Those products require some extensive work, such a model finetuning on proprietary data. Who is going to invest time & money into something like that when OpenAI says right out of the gate they may not support this model for very long?

Basically OpenAI is telegraphing that this is yet another prototype that escaped a lab, not something that is actually ready for use and deployment.

superq• 1 year ago

Complete legal arguments as well. If I was an attorney, I'd love to have a sophisticated LLM write my crib notes for anything I might do or say in the court room, or even the complete direction that I'd take my case. For some cases, that'd be worth almost any price.

nunez2• 1 year ago

RFP automation software has existed for a very long time. Anyone who spends lots of time on RFPs has this.

kristofferR• 1 year ago

"GPT-4.5 is not a frontier model, but it is OpenAI’s largest LLM, improving on GPT-4’s computational efficiency by more than 10x."[1]

I don't get it, it is supposedly much cheaper to run?

[1] https://cdn.openai.com/gpt-4-5-system-card.pdf (page 7, bottom)

refulgentis5• 1 year ago

I speed up my algo that takes a bag-o'-floats by 10x.

If I put 100x floats in my bag-o'-floats, its still 10x slower :(

(extending beyond that point and beyond ELI5: computational efficiency implies multiplying the floats is faster, but you still need the whole bag o' floats, i.e no RAM efficiency gained, so you're still screwed on big-O for the # of GPUs you need to use)

acchow• 1 year ago

It sounds like it's so expensive and the difference in usefulness is so lacking(?)

The claimed hallucination rate is dropping from 61% to 37%. That's a "correct" rate increasing from 29% to 63%.

Double the correct rate costs 15x the price? That seems absurd, unless you think about how mistakes compound. Even just 2 steps in and you're comparing a 8.4% correct rate vs 40%. 3 automated steps and it's 2.4% vs 25%.

einrealist• 1 year ago

And remember, with increasing accuracy, the cost of validation goes up (not even linear).

We expect computers to be right. Its a trust problem. Average users will simply trust the results of LLMs and move on without proper validation. And the way the LLMs are trained to mimic human interaction is not helping either. This will reduce overall quality in society.

Its a different thing to work with another human, because there is intention. A human wants to be correct or to mislead me. I am considering this without even thinking about it.

And I don't expect expert models to improve things, unless the problem space is really simple (like checking eggs for anomalies).

isk517• 1 year ago

30x price bump feels like a attempt to pull in as much money as possible before the bubble bursts.

dr_kiszonka• 1 year ago

To me, it feels like a PR stunt in response to what the competition is doing. OpenAI is trying to show how they are ahead of others, but they price the new model to minimize its use. Potentially, Anthropic et al. also have amazing models that they aren't yet ready to productionize because of costs.

ic4l• 1 year ago

Did they already disable it?

When using `gpt-4.5-preview` I am getting: > Invalid URL (POST /v1/chat/completions)

bawolff• 1 year ago

It sounds like it's so expensive and the difference in usefulness is so lacking(?) they're not even gonna keep serving it in the API for long:

Sounds like an attempt at price descrimination. Sell the expensive version to big companies with big budgets who don't care, sell the cheap version to everyone else. Capture both ends of the market.

OldGreenYodaGPT2• 1 year ago

This is was GPT4 cost when it was released

campers• 1 year ago

The price will come down over time as they apply all the techniques to distill it down to a smaller parameter model. Just like GPT4 pricing came down significantly over time.

osigurdson2• 1 year ago

I don't understand the pricing for cached tokens. It seems rather high for looking up something in a cache.

mmaunder• 1 year ago

But you get higher EQ. /s

muzani2• 1 year ago

For comparison, 3 years ago, the most powerful model out there (GPT-3 davinci) was $60/MTok.

quantadev• 1 year ago

It's crazy expensive because they want to pull in as much revenue as possible as fast as possible before the Open Source models put them outta business.

kla-s• 1 year ago

Well to play the devils advocat, i think this is useful to have, at least for ‘Open’Ai to start off from to apply QLora or similar approximations.

Bonus they could even do some self learning afterwards with the performance improvements DeepSeek just published and it might have more EQ and less hallucinations than starting from scratch…

ie the price might go down big time but there might be significant improvements down the line when starting from such a broad base

dtnewman• 1 year ago

Really depends on your use case. For low value tasks this is way too expensive. But for context, let’s say a court opinion is an average of 6000 words. Let’s say i want to analyze 10 court opinions and pull some information out that’s relevant to my case. That will run about $1.80 per document or $18 total. I wouldn’t pay that just to edify myself, but i can think of many use cases where it’s still a negligible cost, even if it only does 5% better than the 30x cheaper model.

dmvdoug• 1 year ago

You’re also insane if you’re a lawyer trusting gen AI for that. Set aside the fact that people are being caught doing it and judges are clearly getting sick of it (so, it’s a threat to your license). You also have an ethical duty to your client. I really don’t understand lawyers who can sign off on papers without themselves having reviewed the material they’re basing it on. Wild.

Foobar8568• 1 year ago

Someone in another comment said that gpt-4 32k had somewhat the same cost (ok 10% cheaper), what was a pain was more the latency and speed than actual cost given the increase in productivity for our usage.

DonHopkins5• 1 year ago

Maybe they started a really long expensive training session, and Elon Musk's DOGE script kiddies somehow broke in and sabotaged it, so it got disrupted and turned into the Eraserhead baby, but they still want to get it out there for a little while before it died to squeeze all the money out of it as possible, because it was so expensive to train.

https://www.youtube.com/watch?v=ZZ-kI4Qzj9U

DonHopkins5• 1 year ago

GPT 4.5 pricing is insane: Price Input: $75.00 / 1M tokens Cached input: $37.50 / 1M tokens Output: $150.00 / 1M tokens

How many eggs does that include??!

fvv• 1 year ago

usefulness is bound to scope/purpose, even if innovation stops, in 3y (thanks to hw and tuning progress ) when 4o costs 0.1$/M and 4.5 1$/M even being a small improvement ( which is not imo ), you will chose to use 4.5 , exactly like no one now want to use 3.5

UrineSqueegee2• 1 year ago

It's priced like this because it can generate erotica.

mayoosh• 1 year ago

It's also not clear what the definite use case is for this versus other models like o3.

madduci• 1 year ago

Let's see if DeepSeek will make a distillation of this model as well

williamsss• 1 year ago

The performance bump doesn't justify the steep price difference.

From a for profit business lens for OpenAI - I understand pushing the price outside the range of side projects, but this pushes it past start ups.

Excited to see new stuff released past reasoning models in any case. Hope they can improve the price soon.

MagicMoonlight• 1 year ago

I put "hello" into it and it billed me 30p for it. Absolutely unusable, more expensive than realtime voice chat.

FuckButtons• 1 year ago

I suspect this is GPT-5. This is the biggest model they made and they got very little ROI hence the re-branding.

lend000• 1 year ago

My understanding is that o1 is a system built on GPT-4o, so this pricing might explain why o3 (the alleged full version) cost so much money to run in the published benchmark tests [0]. It must be using GPT 4.5 or something similar as the underlying model.

[0] https://arcprize.org/blog/oai-o3-pub-breakthrough

Xiol32• 1 year ago

The example GPT-4.5 answers from the livestream are just... too excitable? Can't put my finger on it, but it feels like they're aimed towards little kids.

jug4• 1 year ago

It made me wonder how much of that was due to the system prompt too.

virgildotcodes• 1 year ago

That presentation was super underwhelming. We got to watch them compare… the vibes? … of 4.5 vs o1.

No wonder Sam wasn’t part of the presentation.

Etheryte2• 1 year ago

And to top it off, it costs $75.00 per 1M vibes.

thomas34298• 1 year ago

Sam tweeted "taking care of my kid in the hospital":

https://x.com/sama/status/1895210655944450446

Let's not assume that he's lying. Neither the presentation nor my short usage via the API blew me away, but to really evaluate it, you'd have to use it longer on a daily basis. Maybe that becomes a possiblity with the announced performance optimizations that would lower the price...

dot1x• 1 year ago

I think it's pretty clear he's a liar in most facets of his life

jug4• 1 year ago

It should’ve just been a web launch without video.

bakugo• 1 year ago

API is literally 5 times more expensive than Claude 3 Opus, and it doesn't even seem to do anything impressive. What's the business strategy here?

hidelooktropic3• 1 year ago

I’m not sure that doing a live stream on this was the right way to go. I would’ve just quietly sent out a press release. I’m sure they have better things on the way.

sebastiennight16• 1 year ago

It is interesting that they are focusing a large part of this release on the model having a higher "EQ" (Emotional Quotient).

We're far from the days of "this is not a person, we do not want to make it addictive" and getting a firm foot on the territory of "here's your new AI friend".

This is very visible in the example comparing 4o with 4.5 when the user is complaining about failing a test, where 4o's response is what one would expect from a "typical AI response" with problem-solving bullets, and 4.5 is sending what you'd expect from a pal over instant messaging.

It seems Anthropic and Grok have both been moving in this direction as well. Are we going to see an escalation of foundation models impersonating "a friendly person" rather than "a helpful assistant"?

Personally I find this worrying and (as someone who builds upon SOTA model APIs) I really hope this behavior is not going to seep into API responses, or will at least be steerable through the system/developer prompt.

famouswaffles• 1 year ago

The whole robotic, monotone, helpful assistant thing was something these companies had to actively hammer in during the post-training stage. It's not really how LLMs will sound by default after pre-training.

I guess they're caring less and less about that effort especially since it hurts the model in some ways like creative writing.

sebastiennight16• 1 year ago

If it's just a different choice during RLHF, I'll be curious to see what are the trade-offs in performance.

The "buddy in a chat group" style answers do not make me feel like asking it for a story will make the story long/detailed/poignant enough to warrant the difference.

I'll give it a try and compare on creative tasks.

turnsout2• 1 year ago

Or maybe they're just getting better at it, or developing better taste. After switching to Claude, I can't go back to ChatGPT's overly verbose bullet-point laden book reports every time I ask a question. I don't think that's pretraining—it's in the way OpenAI approaches tuning and prompting vs Anthropic.

capnrefsmmat2• 1 year ago

Maybe, but I'm not sure how much the style is deliberate vs. a consequence of the post-training tasks like summarization and problem solving. Without seeing the post-training tasks and rating systems it's hard to judge if it's a deliberate style or an emergent consequence of other things.

But it's definitely the case that base models sound more human than instruction-tuned variants. And the shift isn't just vocabulary, it's also in grammar and rhetorical style. There's a shift toward longer words, but also participial phrases, phrasal coordination (with "and" and "or"), and nominalizations (turning adjectives/adverbs into nouns, like "development" or "naturalness"). https://arxiv.org/abs/2410.16107

tmaly2• 1 year ago

I would like to see a humor test. So far, I have not seen any model response that has made me laugh.

sebastiennight16• 1 year ago

The "roast" tools that have popped up (using either DeepSeek or o3-mini) are pretty funny.

Eg. https://news.ycombinator.com/item?id=43163654

AgentME• 1 year ago

My benchmark for this has been asking the model to write some tweets in the style of dril, a popular user who writes short funny tweets. Sometimes I include a few example tweets in the prompt too. Here's an example of results I got from Claude 3 Opus and GPT 4 for this last year: https://bsky.app/profile/macil.tech/post/3kpcvicmirs2v. My opinion is that Claude's results were mostly bangers while GPT's were all a bit groanworthy. I need to try this again with the latest models sometime.

tkgally2• 1 year ago

How does the following stand-up routine by Claude 3.7 Sonnet work for you?

https://gally.net/temp/20250225claudestandup2.html

turnsout2• 1 year ago

If you like absurdist humor, go into the OpenAI playground, select 3.5-Turbo, and dial up the temperature to the point where the output devolves into garbled text after 500 tokens or so. The first ~200 tokens are in the freaking sweet spot of humor.

immibis• 1 year ago

ChatGPT gave me this shell script: https://social.immibis.com/media/7102ac83cf4a200e48dd368938e... (obviously, don't download and execute a random shell script from the internet without reading it first)

I think reading it will make you laugh.

nialv73• 1 year ago

Well yeah, if the llm can keep you engaged and talking, that'll make them a lot more money; compared to if you just use it as a information retrieval tool in which case you are likely to leave after getting what you are looking for.

TheAceOfHearts• 1 year ago

Since they offer a subscription, keeping you engaged just requires them to waste more compute. The ideal case would be that the LLM gives you a one shot correct response using as little compute as possible.

bredren2• 1 year ago

Yes, the "personality" (vibe) of the model is a key qualitative attribute of gpt-4.5.

I suspect this has something to do with shining light on an increased value prop in a dimension many people will appreciate since gains on quantitative comparison with other models were not notable enough to pop eyeballs.

callc• 1 year ago

We're far from the days of "this is not a person, we do not want to make it addictive" and getting a firm foot on the territory of "here's your new AI friend".

That’s a hard nope from me, when companies pull that move. I’ll stick to my flesh and blood humans who still hallucinate but only rarely.

orbital-decay2• 1 year ago

Anthropic pretty much abandoned this direction after Claude 3, and said it wasn't what they wanted [1]. Claude 3.5+ is extremely dry and neutral, it doesn't seem to have the same training.

Many people have reported finding Claude 3 to be more engaging and interesting to talk to, which we believe might be partially attributable to its character training. This wasn’t the core goal of character training, however. Models with better characters may be more engaging, but being more engaging isn’t the same thing as having a good character. In fact, an excessive desire to be engaging seems like an undesirable character trait for a model to have.

[1] https://www.anthropic.com/research/claude-character

Kye2• 1 year ago

It's the opposite incentive to ad-funded social media. One wants to drain your wallet and keep you hooked, the other wants you to spend as little of their funding as possible finding what you're looking for.

sureIy• 1 year ago

I don't know if I fully agree. The input clearly shows the need for emotional support more than "how do I pass this test?" The answer by 4o is comical even if you know you're talking to a machine.

It reminds me of the advice to "not offer solutions when a woman talks about her problems, but just listen."

neuroticnews25• 1 year ago

How could a machine provide emotional support? When I ask questions like this to LLMs, it's always to brainstorm solutions. I get annoyed when I receive fake-attention follow-up questions instead.

I guess there's a trade-off between being human and being useful. But this isn't unique to LLMs, it's similar to how one wouldn't expect a deep personal connection with a customer service professional.

aprilthird20213• 1 year ago

I think it's a good thing because, idk why, I just start tuning out after getting reams and reams of bullet points I'm already not super confident about the truthfulness of

oblio4• 1 year ago

We're far from the days of "this is not a person, we do not want to make it addictive" and getting a firm foot on the territory of "here's your new AI friend".

And soon we'll have the new AI friend recommending Bud Lite™ and turning the beer can with the logo towards you.

CitizenTen• 1 year ago

GPT pro already has already rummored to be 100k users. You think GPT 4.5 will add to that even with the insane costs for corporate users?

cristiancavalli3• 1 year ago

What rumors? I looked and can’t find something to substantiate that #

pepsi-not-coke• 1 year ago

in my experience, o3-mini-high while still unpredictable as it modifies and ignores parts of my code when I specifically tell it not to do so (e.g. "don't touch anything else!") is the best AI coding tool out there, far better than Claude

so Pro is worth it for O3-mini-high

spiderfarmer2• 1 year ago

100k per user perhaps.

andsoitis• 1 year ago

“this isn’t a reasoning model and won’t crush benchmarks.”

— https://x.com/sama/status/1895203654103351462

MaxPock• 1 year ago

One thing that Altman does extremely well is to over-promise and under-deliver.

erulabs• 1 year ago

Finally a scaling wall? This is apparently (based on pricing) using about an order of magnitude more compute, and is only maybe 10% more intelligent. Ideally DeepSeeks optimizations help bring the costs way down, but do any AI researchers want to comment on if this changes the overall shape of the scaling curve?

fpgaminer• 1 year ago

Seems on par with the existing scaling curve. If I had to speculate, this model would have been an internal-only model, but they're releasing it for PR. An optimized version with 99% of the performance for 1/10th the cost will come out later.

j_maffe6• 1 year ago

This is the shittiest PR move I've seen since the AI trend started.

csomar• 1 year ago

We have hit that wall almost 2 years ago with gpt-4. There was clearly no scaling as gpt-4 was already decently smart and if you got x2 smarter you’ll be more capable than anything on the market today. All models doing today (R1 and friends; and Claude) are trying to optimize this local maxima toward generating more useful responses (ie: code when it comes to Claude).

AI, at its current form, is a Deep Seek of compressed knowledge in a 30-50gb of interconnected data. I think we’ll look at this as trying to train networks on corpus of data and expecting them to have a hold of reality. Our brains are trained on “reality” which is not the “real” reality as your vision is limited to the visible spectrum. But if you want a network to behave like a human then maybe give him what a human see.

There is also the possibility that there is a physical limit to intelligence. I don’t see any elephants doing PhDs and the smartest of humans are just a small configuration away from insanity.

killerstorm• 1 year ago

It depends on how you compare.

On a subset of tasks I'm interested in, it's 10x more intelligent than GPT-4. (Note that GPT-4 was in many ways better than 4o.)

It's not a coding champion, but it knows A LOT of stuff, excellent common sense, top quality writing. For me it's like "deep research lite".

I found OpenAI Deep research excellent, but GPT-4.5 might in many cases beat it.

_1tem• 1 year ago

On a subset of tasks I'm interested in, it's 10x more intelligent than GPT-4.

Very intriguing. Care to share an example?

vbezhenar4• 1 year ago

The price is 2x from GPT4. So probably not a decimal order of magnitude.

ein0p• 1 year ago

Now imagine this model (or an optimized/slightly downsized variant thereof) as a base for a "thinking" one.

bparsons• 1 year ago

Are they saying that 4.5 has a 35% hallucination rate? That chart is a bit confusing.

riku_iki• 1 year ago

its on that benchmark, which likely is very challenging.

Seattle3503• 1 year ago

The GPT-1 response to the example prompt "What was the first language?" got a chuckle out of me

aldanor2• 1 year ago

The question being, will we be chuckling at current models responses in 5-10y from now?

joshuamcginnis• 1 year ago

I'm one week in on heavy grok usage. I didn't think I'd say this, but for personal use, I'm considering cancelling my OpenAI plan.

The one thing I wish grok had was more separation of the UI from X itself. The interface being so coupled to X puts me off and makes it feel like a second-hand citizen. I like ChatGPTs minimalist UI.

aldanor2• 1 year ago

Theres grok.com which is standalone and with its own UI

it• 1 year ago

There's also a standalone Grok app at least on iOS.

richard_todd• 1 year ago

I find grok to be the best overall experience for the types of tasks I try to give AI (mostly: analyze pdf, perform and proofread OCR, translate Medieval Latin and Hebrew, remind me how to do various things in python or SwiftUI). ChatGPT/gemini/copilot all fight me occasionally, but grok just tries to help. And the hallucinations aren’t as frequent, at least anecdotally.

fzzzy• 1 year ago

Don't they have a standalone Grok app now? I thought I saw that. [edit] ah some sibling comments mention this as well

andxor• 1 year ago

https://grok.com/

arresin• 1 year ago

They still haven’t released an API for 3

misiti3780• 1 year ago

I canceled my GPT, Grok is incredible.

barfingclouds• 1 year ago

There’s a grok app for iPhone that’s basically the same as ChatGPT/deepseek/mistral/gemini/claude

martin82• 1 year ago

Agree. Grok at the moment is king.

I just wish they had folders (or projects) like OpenAi has...

selalipop• 1 year ago

I've been working on post-training models for tasks that require EQ, so it's validating to see OpenAI working towards that too.

That being said, this is very expensive.

- Input: $75.00 / 1M tokens

- Cached input: $37.50 / 1M tokens

- Output: $150.00 / 1M tokens

One of the most interesting applications of models with higher EQ is personalized content generation, but the size and cost here are at odds with that.

kgeist• 1 year ago

GPT‑4.5 is more succinct and conversational

I wonder why they highlight it as an achievement when they could have simply tuned 4o to be more conversational and less like a bullet-point-style answer machine. They did something to 4o compared to the previous models which made the responses feel more canned.

qgin• 1 year ago

Poossibly, but reports seem to indicate that 4.5 is much more nuanced and thoughtful in its language use. It's not just being shorter and casual as a style, there is a higher amount of "conceptual resolution" within the words being used.

mvdtnz2• 1 year ago

OpenAI doubling down on the American-style therapy-speak instead of focusing on usefulness. No thanks.

wewewedxfgdf2• 1 year ago

I feel like OpenAI is pursuing AGI when Anthropic/Claude is pursuing making AI awesome for practical things like coding.

I only ever using OpenAI's coding now as a double check against Claude.

Does OpenAI have their eyes on the ball?

ls_stats• 1 year ago

I feel like OpenAI is pursuing AGI

I don't think so, the "AGI guy" was Ilya Sutskever, he is gone, he wanted to make OpenAI "less comercial", AGI is just a buzzword for Altmann.

rakejake2• 1 year ago

Right. A good chunk of the "old guard" is now gone - Ilya to SSI, Mira and a bunch of others to a new venture called Thinking Machines, Alec Radford etc. Remains to be seen if OpenAI will be the leader or if other players catch up.

rakejake2• 1 year ago

My usage has come down to mostly Claude (until I run out of free tier quota) and then Gemini. Claude is the best for code and Gemini 2.0 Flash is good enough while also being free (well considering how much data G has hoovered up over the years, perhaps not) and more importantly highly available.

For simple queries like generating shell scripts for some plumbing, or doing some data munging, I go straight to Gemini.

HarHarVeryFunny• 1 year ago

My usage has come down to mostly Claude (until I run out of free tier quota) and then Gemini

Yep, exactly same here.

Gemini 2.0 Flash is extremely good, and I've yet to hit any usage limits with them - for heavy usage I just go to Gemini directly. For "talk to an expert" usage, Claude is hard to beat though.

wayeq• 1 year ago

Claude still can't make real time web searches though for RAG, unless I'm missing something.

resource0x2• 1 year ago

Pursuing AGI? What method do they use to pursue something that no one knows what it is? They will keep saying they are pursuing AGI as long as there's a buyer for their BS.

anti-soyboy4• 1 year ago

Open AI is pursuing bullshit as they realised they can not compete anymore as they fired most of their talent year ago

lblume• 1 year ago

Am I missing something, or do the results not even look that much better? Referring to the output quality, this just seems like a different prompting style and RLHF, not really an improved model at all.

infinet3• 1 year ago

Can it be self-hosted? Many institutions and organizations are hesitant to use AI because concerns of data leaking over chatbot. Open models, on the other hand, can be self-hosted. There is a deepseek arm race in other part of the world. Universities are racing to host their own deepseek. Hospitals, large businesses, local governments, even courts are deploying or showing interest in self-hosting deepseek.

YetAnotherNick2• 1 year ago

Do you know of any university that host Deepseek?

infinet3• 1 year ago

Do you know of any university that host Deepseek?

https://chat.zju.edu.cn

https://chat.sjtu.edu.cn

https://chat.ecnu.edu.cn/html/

To list a few. There are of course many more in China. I won't be surprised if universities in other countries also self-hosting.

moralestapia2• 1 year ago

OpenAI has never released a single model that could be self-hosted.

GPT-2? Maybe not even that one.

taytus2• 1 year ago

Who wants a model that is not reasoning? The older models are just fine.

ksynwa2• 1 year ago

They said this is their last non reasoning model so I'm assuming there is a sunk cost aspect to it.

eightysixfour10• 1 year ago

Seeing OpenAI and Anthropic go different routes here is interesting. It is worth moving past the initial knee jerk reaction of this model being unimpressive and some of the comments about "they spent a massive amount of money and had to ship something for it..."

* Anthropic appears to be making a bet that a single paradigm (reasoning) can create a model which is excellent for all use cases.

* OpenAI seems to be betting that you'll need an ensemble of models with different capabilities, working as a single system, to jump beyond what the reasoning models today can do.

Based on all of the comments from OpenAI, GPT 4.5 is absolutely massive, and with that size comes the ability to store far more factual data. The scores in ability oriented things - like coding - don't show the kind of gains you get from reasoning models but the fact based test, SimpleQA, shows a pretty large jump and a dramatic reduction in hallucinations. You can imagine a scenario where GPT4.5 is coordinating multiple, smaller, reasoning agents and using its factual accuracy to enhance their reasoning, kind of like ruminating on an idea "feels" like a different process than having a chat with someone.

I'm really curious if they're actually combining two things right now that could be split as well, EQ/communications, and factual knowledge storage. This could all be a bust, but it is an interesting difference in approaches none-the-less, and worth considering that OpenAI could be right.

nomel3• 1 year ago

OpenAI seems to be betting that you'll need an ensemble of models with different capabilities, working as a single system, to jump beyond what the reasoning models today can do.

The high level block diagrams for tech always end up converging to those found in biological systems.

eightysixfour10• 1 year ago

Yeah, I don't know enough real neuroscience to argue either side. What I can say is I feel like this path is more like the way that I observe that I think, it feels like there are different modes of thinking and processes in the brain, and it seems like transformers are able to emulate at least two different versions of that.

Once we figure out the frontal cortex & corpus callosum part of this, where we aren't calling other models over APIs instead of them all working in the same shared space, I have a feeling we'll be on to something pretty exciting.

sebastiennight16• 1 year ago

* OpenAI seems to be betting that you'll need an ensemble of models with different capabilities, working as a single system, to jump beyond what the reasoning models today can do.

Seems inaccurate as their most recent claim I've seen is that they expect this to be their last non-reasoning model, and are aiming to provide all capacities together in the future model releases (unifying the GPT-x and o-x lines)

See this claim on TFA:

We believe reasoning will be a core capability of future models, and that the two approaches to scaling—pre-training and reasoning—will complement each other.

eightysixfour10• 1 year ago

From Sam's twitter:

After that, a top goal for us is to unify o-series models and GPT-series models by creating systems that can use all our tools, know when to think for a long time or not, and generally be useful for a very wide range of tasks.
In both ChatGPT and our API, we will release GPT-5 as a system that integrates a lot of our technology, including o3. We will no longer ship o3 as a standalone model.

You could read this as unifying the models or building a unified systems which coordinate multiple models. The second sentence, to me, implies that o3 will still exist, it just won't be standalone, which matches the idea I shared above.

wongarsu• 1 year ago

Or the other way around: smaller reasoning models that can call out to GPT-4.5 to get their facts right.

eightysixfour10• 1 year ago

Maybe, I’m inclined to think OpenAI believes the way I laid it out though, specifically because of their focus on communication and EQ in 4.5. It seems like they believe the large, non-reasoning model, will be “front of house.”

Or they’ll use some kind of trained router which sends the request to the one it thinks it should go to first.

jstummbillig3• 1 year ago

It can never be just reasoning, right? Reasoning is the multiplier on some base model, and surely no amount of reasoning on top of something like gpt-2 will get you o1.

This model is too expensive right now, but as compute gets cheaper — and we have to keep in mind, that it will — having a better base to multiply with will enable things that just more thinking won't.

eightysixfour10• 1 year ago

You can try for yourself with the distilled R1's that Deepseek released. The qwen-7b based model is quite impressive for its size and it can do a lot with additional context provided. I imagine for some domains you can provide enough context and let the inference time eventually solve it, for others you can't.

throw234234234• 1 year ago

Anthropic appears to be making a bet that a single paradigm (reasoning) can create a model which is excellent for all use cases.

I don't think that is their primary motivation. The announcement post for Claude 3.7 was all about code which doesn't seem to imply "all use cases". Code this, new code tool that, telling customers that they look forward to what they build, etc. Very little mention of other use cases on the new model announcement at all. Their usage stats they published are telling - 80%+ or more of queries to Claude are all about code. i.e. I actually think while they are thinking of other use cases; they see the use case of code specifically as the major thing to optimize for.

OpenAI, given its different customer base and reach, is probably aiming for something more general.

IMO they all think that you need an "ensemble" of models with different capabilities to optimise for different use cases. Its more about how much compute resources each company has and what they target with those resources. Anthrophic I'm assuming has less compute resources and a narrower customer base so it economically may make sense to optimise just for that.

eightysixfour10• 1 year ago

That's possible, my counter point would be that if that was the case Anthropic would have built a smaller reasoning model instead of doing a "full" Claude. Instead, they built something which seems to be flexible across different types of responses.

Only time will tell.

protocolture• 1 year ago

Ever since those kids demo'd their fact checking engine here, which was just Input -> LLM -> Fact Database -> LLM -> LLM -> Output I have been betting that it will be advantageous to move in this general direction.

ilrwbwrkhv• 1 year ago

Question: explain thread first macros in clojure in a short paragraph

GPT 4.5:

In Clojure, the *thread-first macro* (`->`) is used to improve code readability by threading a value through a series of function calls. It takes an initial value and passes it as the first argument to the next function, then continues threading the result through subsequent functions. This eliminates deeply nested expressions and makes the code more readable.

For example,

```clojure (-> 5 inc (* 2) str) ;; Equivalent to (str (* 2 (inc 5))) → "12" ```

Here, `5` is incremented to `6`, then multiplied by `2` to get `12`, and finally converted to a string.

---

Deepseek v3:

In Clojure, the `->>` (thread-last) and `->` (thread-first) macros are used to improve code readability by allowing a more linear, pipeline-like flow of operations. The `->` (thread-first) macro takes an initial value and threads it as the first argument through a series of forms. For example, `(-> x (f) (g))` translates to `(g (f x))`. This is particularly useful when working with nested function calls, as it flattens the structure and makes the sequence of transformations clearer. It’s commonly used with functions that take the threaded value as their first argument, such as collection operations or Java interop methods.

datadrivenangel• 1 year ago

They tried, but doesn't look better?

DaveMcMartin• 1 year ago

This feels more like a release they pushed out to keep the "hype" alive rather than something they were eager to share. Honestly, the results don’t seem all that impressive, and considering the price, it just doesn’t feel worth it.

bla3• 1 year ago

I wonder if we're starting to see the effects of the mass exodus a while ago.

BoredPositron• 1 year ago

The announcements early on were relatively sincere and technical with papers and nice pages explaining the new models in easy language and now we get this marketing garbage. Probably the fastest enshitification I’ve seen.

sky2224• 1 year ago

Honestly, the most astounding part of this announcement is their comparison to o3-mini with QA prompts.

EIGHTY PERCENT hallucination rate? Are you kidding me?

I get that the model is meant to be used for logic and reasoning, but nowhere does OpenAI make this explicitly clear. A majority of users are going to be thinking, "oh newer is better," and pick that.

jug4• 1 year ago

Yeah it was an abysmal result (any 50%+ hallucination result in that bench is pretty bad) and worse than o1-mini in the SimpleQA paper. On that topic, Sonnet 3.5 ”Old” hallucinates less than GPT-4.5, just for a bit of added perspective here.

krackers2• 1 year ago

Very nice catch, I was under the impression that o3-mini was "as good" as o1 on all dimensions. Seems the takeaway is that any form of quantization/distillation ends up hurting factual accuracy (but not reasoning performance), and there are diminishing returns to reducing hallucinations by model-scaling or RLHF'ing. I guess then that other approaches are needed to achieve single-digit "hallucination" rates. All of wikipedia compresses down to < 50GB though, so it's not immediately clear that you can't have good factual accuracy with a small sparse model

freediver• 1 year ago

The results for GPT - 4.5 are in for Kagi LLM benchmark too.

It does crush our benchmark - time to make new? ;) - with performance similar of that of reasoning models. It does come at a great price both in cost and speed.

A monster is what they created. But looking at the tasks it fails, some of them my 9 year old would solve. Still in this weird limbo space of super knowledge and low intelligence.

May be remembered as the last the last of the 'big ones', can't imagine this will be a path for the future.

https://help.kagi.com/kagi/ai/llm-benchmark.html

theodorthe5• 1 year ago

If Gemini 2 is the top in your benchmark, make sure to re-check your benchmark.

shawabawa32• 1 year ago

Gemini 2 pro is actually very impressive (maybe not for coding, haven't used it for that)

Flash is pretty garbage but cheap

istjohn2• 1 year ago

Gemini 2.0 Pro is quite good.

aoeusnth13• 1 year ago

Gemini 2 pro is pretty strong actually.

wendyshu• 1 year ago

Why don't you have Grok?

mhh__• 1 year ago

No api for grok 3 might be why

mjirv• 1 year ago

Do you have results for gpt-4? I’d be very interested in seeing the lift here from their last “big one”.

simonw4• 1 year ago

If you want to try it out via their API you can run it through my LLM tool using uvx like this:

  uvx --with 'https://github.com/simonw/llm/archive/801b08bf40788c09aed6175252876310312fe667.zip' \
    llm -m gpt-4.5-preview 'impress me'

You may need to set an API key first, either with `export OPENAI_API_KEY='xxx'` or using this command to save it to a file:

  uvx llm keys set openai
  # paste key here

Or this to get a chat session going:

  uvx --with 'https://github.com/simonw/llm/archive/801b08bf40788c09aed6175252876310312fe667.zip' \
    llm chat -m gpt-4.5-preview

I'll probably have a proper release out later today. Details here: https://github.com/simonw/llm/issues/795

ashu14613• 1 year ago

Just curious, does this stream the output or renders all at once ?

simonw4• 1 year ago

It streams the output. See animated demo here (bottom image on the page) https://simonwillison.net/2025/Feb/27/introducing-gpt-45/

synapsomorphy• 1 year ago

Claude 3.6 (new 3.5) and 3.7 non-reasoning are much better at pretty much everything, and much cheaper. What's Anthropic's secret sauce?

taytus2• 1 year ago

They ship more focused on their mission than OpenAI.

moralestapia2• 1 year ago

Huh?

Post benchmark links.

film42• 1 year ago

I think it's a classic expectations problem. OpenAI is neither _open_ nor is it releasing an _AGI_ model in the near future. But when you see a new major model drop, you can't help but ask, "how close is this to the promise of AGI they say is just around the corner?" Not even close. Meanwhile Anthropic is keeping their heads down, not playing the hype game, and letting the model speak for itself.

anothermathbozo• 1 year ago

Anthropic’s CEO said their technology would end all disease and expand our lifespans to 200 years. What on earth do you mean they’re not playing the hype game?

jampa3• 1 year ago

First impression of GPT-4.5:

1. It is very very slow, for some applications where you want real time interactions is just not viable, the text attached below took 7s to generate with 4o, but 46s with GPT4.5

2. The style it writes is way better: it keeps the tone you ask and makes better improvements on the flow. One of my biggest complaints with 4o is that you want for your content to be more casual and accessible but GPT / DeepSeek wants to write like Shakespeare did.

Some comparisons on a book draft: GPT4o (left) and GPT4.5 (green). I also adjusted the spacing around the paragraphs, to better diff match. I still am wary of using ChatGPT to help me write, even with GPT 4.5, but the improvement is very noticeable.

https://i.imgur.com/ogalyE0.png

remus• 1 year ago

It is very very slow

Could that be partially due to a big spike in demand at launch?

jampa3• 1 year ago

Possibly, repeating the prompt I got a much higher speed, taking 20s on average now, which is much more viable. But that remains to be seen when more people start using this version in production.

jedberg• 1 year ago

Oh yeah, that right side version is WAY better, and sounds much more like a human.

MichaelZuo4• 1 year ago

How does it compare with o1 and o3 preview?

jampa3• 1 year ago

o3 is okay for text checking but has issues following the prompt correctly, same as o1 and DeepSeek R1, I feel that I need to prompt smaller snippets with them.

Here is the o3 vs a new run of the same text in GPT 4.5

https://www.diffchecker.com/ZEUQ92u7/

FergusArgyll• 1 year ago

I opened your link in a new tab and looked at it a couple minutes later. By then I forgot which was o and which was .5

I honestly couldn't decide which I prefer

niek_pas• 1 year ago

I definitely prefer the 4.5, but that might just be because it sounds 'less like ChatGPT', ironically.

rl34• 1 year ago

1. It is very very slow, ... below took 7s to generate with 4o, but 46s with GPT4.5

This is positively luxurious by o1-pro standards which I'd say average 5 minutes. That said I totally agree even ~45s isn't viable for real-time interactions. I'm sure it'll be optimized.

Of course, my comparing it to the highest-end CoT model in [publicly-known] existence isn't entirely fair since they're sort of apples and oranges.

philomath_mn3• 1 year ago

I paid for pro to try `o1-pro` and I can't seem to find any use case to justify the insane inference time. `o3-mini-high` seems to do just as well in seconds vs. minutes.

thfuran• 1 year ago

One of my biggest complaints with 4o is that you want for your content to be more casual and accessible but GPT / DeepSeek wants to write like Shakespeare did.

Well, maybe like a Sophomore's bumbling attempt to write like Shakespeare.

ChiefNotAClue• 1 year ago

Right side, by a large margin. Better word choice and more natural flow. It feels a lot more human.

rossant• 1 year ago

Is there really no way to prompt GPT4o to use a more natural and informal tone matching GPT4.5's?

kristianp• 1 year ago

How do the two versions match so closely? They have the same content in each paragraph, just worded slightly differently. I wouldn't expect them to write paragraphs that match in size and position like that.

princealiiiii• 1 year ago

Honestly, feels like a second LLM just reworded the response on the left-side to generate the right-side response.

throwaway3141554• 1 year ago

If you use the "retry" functionality in ChatGPT enough, you will notice this happens basically all the time.

kumarm• 1 year ago

Thank you. This is the best example of comparison I have seen so far.

reassess_blind• 1 year ago

What’s the deal with Imgur taking ages to load? Anyone else have this issue in Australia? I just get the grey background with no content loaded for 10+ seconds every time I visit that bloated website.

stevage3• 1 year ago

Ok for me here in aus

elliotto• 1 year ago

This website sucks but successfully loaded from Aus rn on my phone. It's full of ads - possibly your ad blocker is killing it?

osigurdson2• 1 year ago

I'm wondering if generative AI will ultimately result in a very dense / bullet form style of writing. What we are doing now is effectively this:

bullet_points' = compress(expand(bullet_points))

We are impressed by lots of text so must expand via LLM in order to impress the reader. Since the reader doesn't have time or interest to read the content they must compress it back into bullet points / quick summary. Really, the original bullet points plus a bit more thinking would likely be a better form of communication.

anon373839• 1 year ago

That’s what Axios does. For ordinary events coverage, it’s a great style.

not_a_bot_4sho• 1 year ago

I'm reminded of this great comic

https://marketoonist.com/2023/03/ai-written-ai-read.html

dyauspitr• 1 year ago

Imgur might be the worst image hosting site I’ve ever experienced. Any interaction with that page results in switching images and big ads and they hijack the back button. Absolutely terrible. How far they’ve fallen from when it first began.

bradley13• 1 year ago

I use 4o mostly in German, so YMMV. However, I find a simple prompt controls the tone very well. "This should be informal and friendly", or "this should be formal and business-like".

muzani2• 1 year ago

In my experience, Gemini Flash has been the best at writing, and GPT 3.5 onwards has been terrible.

GPT-3 and GPT-2 were actually remarkably good at it, arguably better than a skilled human. I had a bit of fun ghostwriting with these and got a little fan base for a while.

It seems that GPT-4.5 is better than 4 but it's nowhere near the quality of GPT-3 davinci. Davinci-002 has been nerfed quite a bit, but in the end it's $2/MTok for higher quality output.

It's clear this is something users want, but OpenAI and Anthropic seem to be going in the opposite direction.

vessenes• 1 year ago

Similar reaction here. I will also note that it seems to know a lot more about me than previous models. I’m not sure if this is a broader web crawl, more space in the model, or more summarization of our chats or a combination, but I asked it to psychoanalyze a problem I’m having in the style of Jacques lacan and it was genuinely helpful and interesting, no interview required first; it just went right at me.

To borrow an iain banks word, the “fragre” def feels improved to me. I think I will prefer it to o1 pro, although I haven’t really hammered on it yet.

mchusma• 1 year ago

wow, openai really missed here. Reading the blog I thought like a minor, incremental minor catch up release for 4o. I thought "wow maybe this is cheaper than 4o so it will offset the pricing difference between this and something like Claude Sonnet 3.7 or Gemini 2.0 Flash both of which performs better. But its like 20x-100x more expensive!

In other words, these performance stats with Gemini 2.0 Flash pricing looks reasonable. At these prices, zero usecases for anyone I think. This is a dead on arrival model.

jasonjmcghee2• 1 year ago

This is such as confusing release / announcement.

It seems clearly worse than Claude Sonnet 3.7, yet costs 30x as much?

What's going on with their pricing? I misread it as $7.5/M input and that that was very overpriced... then realized it was 10x that much!

ashu14613• 1 year ago

Is it worse than clause sonnet with reasoning enabled or disabled ?

moffkalast• 1 year ago

OpenAI will do literally anything but ship GPT-5.

saghm• 1 year ago

This is probably a dumb question, but are we just gonna be stuck on always having X.5 versions of GPT forever? If there's never an X.0, it feels like it's basically meaningless.

xd1936• 1 year ago

The gas costs $3.59⁹

dragonwriter• 1 year ago

There was 3, 3.5, 4, 4o, and then 4.5.

There are things to complain about in that, but “its always X.5 and never X.0” doesn't really seem to be one of them.

JohnMakin2• 1 year ago

Their example for 4.5 on "having a tough time":

I’m going through a tough time after failing a test > Aw, I'm really sorry to hear that. Failing a test can feel pretty tough and discouraging, but remember, it's just one moment—not a reflection of who you are or your capabilities. Want to talk about what happened, or do you just need a distraction? I'm here either way.

vs 4o:

I’m going through a tough time after failing a test >I'm really sorry to hear that you're going through a tough time. Here are a few >things you might consider: >1. Reflect and Understand: Try to analyze what might have gone wrong. Was it a >lack of preparation, time management during the test, or perhaps >misunderstanding the material? Understanding the root cause can help you address >it in the future. >2. Seek Support: Talk to someone you trust—friends, family, or a school >counselor. They can offer a different perspective and sometimes just talking >about your feelings can be alleviating. >3. Speak with Your Teacher or Professor: They can provide insights into where >you might have gone wrong and offer advice on how to improve. They might also >offer opportunities for make-up work or extra credit. >4. Develop a Study Plan: Based on your reflections, create a plan to improve >your study habits. This might include setting regular study times, breaking down >material into manageable portions, or finding new study methods that work better >for you. >5. Stay Positive and Forgive Yourself: Mistakes and setbacks are a part of life. >It’s important to be kind to yourself and recognize that failure is a stepping >stone to success. >6. Focus on the Bigger Picture: Remember that one test is just one part of your >educational journey. There will be many more opportunities to do well. >If you need further support or resources, consider reaching out to educational >support services at your institution, or mental health resources if you’re >feeling particularly overwhelmed. You’re not alone in this, and things can get >better with time and effort.

Is it just me or is the 4o response insanely better? I'm not the type of person to reach for a LLM for help about this kind of thing, but if I were, the 4o respond seems vastly better to the point I'm surprised they used that as their main "EQ" example.

IMTDb• 1 year ago

4o has a very strong artificial vibe. It feels a bit "autistic" (probably a bad analogy but couldn't find a better word to describe what I mean): you feel bad ? must say sorry then give a TODO list on how to feel better.

4.5 still feels a bit artificial but somehow also more emotionally connected. It removed the weird "bullet point lists of things to do" and focused on the emotional part; which is also longer than 4o

If I am talking to a human I would definitely expect him/her to react more like 4.5 than like 4o. If the first sentence that comes out of their mouth after I explain them that I feel bad is "here is a list of things you might consider", I will find it strange. We can reach that point but it's usually after a bit more talk; human kinda need that process, and it feels like 4.5 understands that better than 4o.

Now of course which one is "better" really depends on the context; what you expect of the model and how you intend to use is. Until now every single OpenAI update on the main series has always been a strict improvement over the previous model. Cost aside, there wasn't really any reason to keep using 3.5 when 4 got released. This is not the case here; even assuming unlimited money you still might wanna select 4o in the dropdown sometimes instead of 4.5.

buu700• 1 year ago

I had a similar gut reaction, but on reflection I think 4.5's is actually the better response.

On one hand, the response from 4.5 seems pretty useless to me, and I can't imagine a situation in which I would personally find value in it. On the other hand, the prompt it's responding to is also so different from how I actually use the tool that my preferences aren't super relevant. I would never give it a prompt that didn't include a clear question or direction, either explicitly or implicitly from context, but I can imagine that someone who does use it that way would actually be looking for something more in line with the 4.5 response than the 4o one. Someone who wanted the 4o response would likely phrase the prompt in a way that explicitly seeks actionable advice, or if they didn't initially then they would in a follow-up.

Where I really see value in the model being capable of that type of logic isn't in the ChatGPT use case (at least for me personally), but in API integrations. For example, customer service agents being able to handle interactions more delicately is obviously useful for a business.

All that being said, hopefully the model doesn't have too many false positives on when it should provide an "EQ"-focused response. That would get annoying pretty quickly if it kept happening while I was just trying to get information or have it complete some task.

fauigerzigerk• 1 year ago

I think both responses are bizarre and useless. Is there a single person on earth who wouldn't ask questions like "what kind of test?", "why do you think you failed?", "how did you prepare for the test?" before giving advice?

torginus4• 1 year ago

My 2 cents (disclaimer: I am talking out of my ass) here is why GPTs actually suck at fluid knowledge retrievel (which is kinda their main usecase, with them being used as knowledge engines) - they've mentioned that if you train it on 'Tom Cruise was born July 3, 1962', it won't be able to answer the question "Who was born on July 3, 1962", if you don't feed it this piece of information. It can't really internally corellate the information it has learned, unless you train it to, probably via synthethic data, which is what OpenAI has probably done, and that's the information score SimpleQA tries to measure.

Probably what happened, is that in doing so, they had to scale either the model size or the training cost to untenable levels.

In my experience, LLMs really suck at fluid knowledge retrieval tasks, like book recommendation - I asked GPT4 to recommend me some SF novels with certain characteristics, and what it spat out was a mix of stuff that didn't really match, and stuff that was really reaching - when I asked the same question on Reddit, all the answers were relevant and on point - so I guess there's still something humans are good for.

Which is a shame, because I'm pretty sure relevant product recommendation is a many billion dollar business - after all that's what Google has built it's empire on.

woah• 1 year ago

Perhaps you could use LLMs in a list ranking context to generate your scifi recommendations https://github.com/noperator/raink?tab=readme-ov-file

staticman2• 1 year ago

You make a good point: I think these LLM's have a strong bias towards recommending the most popular things in pop culture since they really only find the most likely tokens and report on that.

So while they may have a chance of answering "What is this non mainstream novel about" they may be unable to recommend the novel since it's not a likely series of tokens in response to a request for a book recommendation.

torginus4• 1 year ago

That's really interesting - just made me think about some AI guy at Twitter (when it was called that) talking about how hard it is to create a recommender system that doesn't just flood everyone with what's popular righr now. Since LLMs are neural networks as well, maybe the recommendation algorithms they learn suffer from the same issues

vel0city2• 1 year ago

An LLM on its own isn't necessarily great for fluid knowledge retrieval, as in directly from its training data. But they're pretty good when you add RAG to it.

For instance, asking Copilot "Who was born on July 3, 1962" gave the response:

One notable person born on July 3, 1962, is Tom Cruise, the famous American actor known for his roles in movies like Risky Business, Jerry Maguire, and Rain Man.
Are you a fan of his work?

It cited this page:

https://www.onthisday.com/date/1962/july/3

suddenlybananas2• 1 year ago

Wow it googled the date!

youssefabdelm• 1 year ago

Yep. I've often said RLHF'd LLMs seem to be better at recognition memory than recall memory.

GPT-4o will never offhand, unprompted and 'unprimed', suggest a rare but relevant book like Shinichi Nakazawa's "A Holistic Lemma of Science" but a base model Mixtral 8x22B or Llama 405B will. (That's how I found it).

It seems most of the RLHF'd models seem biased towards popularity over relevance when it comes to recall. They know about rare people like Tyler Volk... but they will never suggest them unless you prime them really heavily for them.

Your point on recommendations from humans I couldn't agree more with. Humans are the OG and undefeated recommendation system in my opinion.

jefffoster• 1 year ago

Does anyone have any intuition about the how reasoning improves based on the strength of the underlying model?

I’m wondering whether this seemingly underwhelming bump on 4o magnifies when/if reasoning is added.

porridgeraisin2• 1 year ago

It is possible to understand the mechanism once you drop the anthropomorphisms.

Each token output by an LLM involves one pass through the next-word predictor neural network. Each pass is a fixed amount of computation. Complexity theory hints to us that the problems which are "hard" for an LLM will need more compute than the ones which are "easy". Thus, the only mechanism through which an LLM can compute more and solve its "hard" problems is by outputting more tokens.

You incentivise it to this end by human-grading its outputs ("RLHF") to prefer those where it spends time calculating before "locking in" to the answer. For example, you would prefer the output

  Ok let's begin... statement1 => statement2 ... Thus, the answer is 5

over

  The answer is 5. This is because....

since in the first one, it has spent more compute before giving the answer. You don't in any way attempt to steer the extra computation in any particular direction. Instead, you simply reinforce preferred answers and hope that somewhere in that extra computation lies some useful computation.

It turned out that such hope was well-placed. The DeepSeek R1-Zero training experiment showed us that if you apply this really generic form of learning (reinforcement learning) without _any_ examples, the model automatically starts outputting more and more tokens i.e "computing more". DeepseekMath was also a model trained directly with RL. Notably, the only signal given was whether the answer was right or not. No attention was paid to anything else. We even ignore the position of the answer in the sequence that we cared about before. This meant that it was possible to automatically grade the LLM without a human in the loop (since you're just checking answer == expected_answer). This is also why math problems were used.

All this is to say, we get the most insight on what benefit "reasoning" adds by examining what happened when we applied it without training the model on any examples. Deepseek R1 actually uses a few examples and then does the RL process on top of that, so we won't look at that.

Reading the DeepseekMath paper[1], we see that the authors posit the following:

  As shown in Figure 7, RL enhances Maj@K’s performance but not Pass@K. These
  findings indicate that RL enhances the model’s overall performance by rendering
  the output distribution more robust, in other words, it seems that the
  improvement is attributed to boosting the correct response from TopK rather 
  than the enhancement of fundamental capabilities.

For context, Maj@K means that you mark the output of the LLM as correct only if the majority of the many outputs you sample are correct. Pass@K means that you mark it as correct even if just one of them is correct.

So to answer your question, if you add an RL-based reasoning process to the model, it will improve simply because it will do more computation, of which a so-far-only-empirically-measured portion helps get more accurate answers on math problems. But outside that, it's purely subjective. If you ask me, I prefer claude sonnet for all coding/swe tasks over any reasoning LLM.

[1] https://arxiv.org/pdf/2402.03300

npinsker• 1 year ago

Thanks for a well-written and clear explanation!

zone4112• 1 year ago

It significantly improves upon GPT-4o on my Extended NYT Connections Benchmark. 22.4 -> 33.7 (https://github.com/lechmazur/nyt-connections).

zone4112• 1 year ago

I ran three more of my independent benchmarks:

- Improves upon GPT-4o's score on the Short Story Creative Writing Benchmark, but Claude Sonnets and DeepSeek R1 score higher. (https://github.com/lechmazur/writing/)

- Improves upon GPT-4o's score on the Confabulations/Hallucinations on Provided Documents Benchmark, nearly matching Gemini 1.5 Pro (Sept) as the best-performing non-reasoning model. (https://github.com/lechmazur/confabulations)

- Improves upon GPT-4o's score on the Thematic Generalization Benchmark, however, it doesn't match the scores of Claude 3.7 Sonnet or Gemini 2.0 Pro Exp. (https://github.com/lechmazur/generalization)

I should have the results from the multi-agent collaboration, strategy, and deception benchmarks within a couple of days. (https://github.com/lechmazur/elimination_game/, https://github.com/lechmazur/step_game and https://github.com/lechmazur/goods).

j_bum4• 1 year ago

Honest question for you: are these puzzles actually a good way to test the models?

The answers are certainly in the training set, likely many times over.

I’d be curious to see performance on Bracket City, which was featured here on HN yesterday.

anotherpaulg• 1 year ago

GPT-4.5 Preview scored 45% on aider's polyglot coding benchmark [0]. OpenAI describes it as "good at creative tasks" [1], so perhaps it is not primarily intended for coding.

  65% Sonnet 3.7, 32k think tokens (SOTA)
  60% Sonnet 3.7, no thinking
  48% DeepSeek V3
  45% GPT 4.5 Preview <===
  27% ChatGPT-4o
  23% GPT-4o

[0] https://aider.chat/docs/leaderboards/

[1] https://platform.openai.com/docs/models#gpt-4-5

doctoboggan3• 1 year ago

I was waiting for your comment and wow... that's bad.

I guess they are ceding the LLMs for coding market to Anthropic? I remember seeing an industry report somewhere and it claimed software development is the largest user of LLMs, so it seems weird to give up in this area.

I_am_tiberius2• 1 year ago

I assume they go all in "the new google" direction. Embedded ads coming soon I guess in the free version (chat.com).

Workaccount23• 1 year ago

4.5 lies on a different path than their STEM models.

o3-mini is an extremely powerful coding model and unquestionably is in the same league as 3.7. o3 is still the top stem overall model.

icemelt8• 1 year ago

they are trying to copy Grok 3

smcleod• 1 year ago

GPT 4.5 is insanely over price, it makes Anthropic look affordable!

shshahshsusus• 1 year ago

brief and detailed summaries by chatgpt (4o):

Brief Summary (40-50 words)

OpenAI’s GPT-4.5 is a research preview of their most advanced language model yet, emphasizing improved pattern recognition, creativity, and reduced hallucinations. It enhances unsupervised learning, has better emotional intelligence, and excels in writing, programming, and problem-solving. Available for ChatGPT Pro users, it also integrates into APIs for developers.

Detailed Summary (200 words)

OpenAI has introduced *GPT-4.5*, a research preview of its most advanced language model, focusing on *scaling unsupervised learning* to enhance pattern recognition, knowledge depth, and reliability. It surpasses previous models in *natural conversation, emotional intelligence (EQ), and nuanced understanding of user intent*, making it particularly useful for writing, programming, and creative tasks.

GPT-4.5 benefits from *scalable training techniques* that improve its steerability and ability to comprehend complex prompts. Compared to GPT-4o, it has a *higher factual accuracy and lower hallucination rates*, making it more dependable across various domains. While it does not employ reasoning-based pre-processing like OpenAI o1, it complements such models by excelling in general intelligence.

Safety improvements include *new supervision techniques* alongside traditional reinforcement learning from human feedback (RLHF). OpenAI has tested GPT-4.5 under its *Preparedness Framework* to ensure alignment and risk mitigation.

*Availability*: GPT-4.5 is accessible to *ChatGPT Pro users*, rolling out to other tiers soon. Developers can also use it in *Chat Completions API, Assistants API, and Batch API*, with *function calling and vision capabilities*. However, it remains computationally expensive, and OpenAI is evaluating its long-term API availability.

GPT-4.5 represents a *major step in AI model scaling*, offering *greater creativity, contextual awareness, and collaboration potential*.

ripped_britches• 1 year ago

Obviously it’s expensive and still I would prefer a reasoning model for coding.

However for user facing applications like mine, this is an awesome step in the right direction for EQ / tone / voice. Obviously it will get distilled into cheaper open models very soon, so I’m not too worried about the price or even tokens per second.

mkaic• 1 year ago

In a hilarious act of accidental satire, it seems that the AI-generated audio version of the post has a weird glitch/mispronunciation within the first three words — it struggles to say "GPT-4.5".

tantalor• 1 year ago

My experience exactly:

1. Open the page

2. Click "Listen to article"

3. Check if I'm having a stroke

4. Close tab

Dear openai: try hiring some humans

a-arbabian• 1 year ago

Common issue with TTS models right now. I use ElevenLabs to dictate articles while I commute and it has a stroke on decimals and symbols.

advael• 1 year ago

It's sad that all I can think about this is that it's just another creep forward of the surveillance oligarchy

I really used to get excited about ML in the wild and while there are much bigger problems right now it still makes me sad to have become so jaded about it

i_love_retros2• 1 year ago

Anyone really finding ai useful for coding?

I'm finding it to make things up, get things wrong, ignore things I ask.

Def not worried about losing my job to it.

i_love_retros2• 1 year ago

It gets confused if I give it 3 files - how is it going to scan a whole codebase and disparate systems and make correct changes.

Pah! Don't believe the hype.

twistslider• 1 year ago

I played around with Claude Code today, first time I've ever really been impressed by AI for coding.

Tasked it with two different things, refactoring a huge function of around ~400 lines and creating some unit tests split into different files. The refactor was done flawlessly. The unit tests almost, only missed some imports.

All I did was open it in the root of my project and prompt it with the function names. It's a large monolithic solution with a lot of subprojects. It found the functions I was talking about without me having to clarify anything. Cost was about $2.

SkyPuncher• 1 year ago

Yes, massively.

There’s a learning curve to it, but it’s worth literally every penny I spend on API calls.

At worst, I’m no faster. At best, it’s easily a 10x improvement.

For me, one of the biggest benefits is talking about coding in natural language. It lowers my mental low and keeps me in a mental space where I’m more easily able to communicate with stakeholders holders.

feznyng• 1 year ago

Really great for quickly building features but you have to be careful about how much context you provide i.e. spoonfeed it exactly the methods, classes, files it needs to do whatever you're asking for (especially in a large codebase). And when it seems to get confused, reset history to free up the context window.

That being said there are definite areas where it shines (cookie cutter UI) and places where it struggles. It's really good at one-shotting React components and Flutter widgets but it tends to struggle with complicated business logic like sync engines. More straightforward backend stuff like CRUD endpoints are definitely doable.

aurareturn3• 1 year ago

Yes, it helps me write SQL queries in seconds that I otherwise would spend days on or give up completely.

antirez• 1 year ago

In many ways I'm not an OpenAI fan (but I need to recognize their many merits). At the same time, I believe people are missing what they tried to do with GPT 4.5: it was needed and important to explore the pre-training scaling law in that direction. A gift to science, however selfist it could be.

throwaway3141554• 1 year ago

A gift to science

This is hardly recognizable as science.

edit: Sorry, didn't feel this was a controversial opinion. What I meant to say was that for so-called science, this is not reproducible in any way whatsoever. Further, this page in particular has all the hallmarks of _marketing_ copy, not science.

Sometimes a failure is just a failure, not necessarily a gift. People could tell scaling wasn't working well before the release of GPT 4.5. I really don't see how this provides as much insight as is suggested.

Deepseek's models apparently still compare favorably with this one. What's more they did that work with the constraint of having _less_ money, not so much money they could run incredibly costly experiments that are likely to fail. We need more of the former, less of the latter.

kadushka• 1 year ago

People could tell scaling wasn't working well before the release of GPT 4.5

Who could tell? Who has tried scaling up to this level?

vbezhenar4• 1 year ago

People could tell scaling wasn't working well before the release of GPT 4.5.

Different people tell different things all the time. That's not science. Experiment is science.

tacet• 1 year ago

if i understand correctly your argument, then i would say that it is very recognizable as science

People could tell scaling wasn't working well before the release of GPT 4.5

Yes, on quick glance it seems so from 2020 openai research into scaling laws.

Scaling apparently didn't work well, so the theory about scaling not working well failed to be falsified. It's science.

wewewedxfgdf2• 1 year ago

GPT-2 was laugh out loud funny, rolling on the ground funny.

I miss that - newer LLMs seem to have lost their sense of humor.

On the other hand GPT-2's funny stories often veered into murdering everyone in the story and committing heinous crimes but that was part of the weird experience.

kossTKR• 1 year ago

Totally agree, i think the gargantuan hidden pre prompts, censorship through reinforcement learning and whatever has killed most creativity.

The newer models are incredible, but the tone is just soul sucking even when it tries to be "looser" in the later iterations.

krackers2• 1 year ago

Sydney is a glimpse at what an "unlobotomized" GPT-4 model would have been like.

lostmsu• 1 year ago

https://hn-wrapped.kadoa.com/wewewedxfgdf

I_am_tiberius2• 1 year ago

Not available in my Pro plan.

highfrequency2• 1 year ago

Overall take seems to be negative in the comments. But I see potential for a non-reasoning model that makes enough subtle tweaks in its tone that it is enjoyable to talk to instead of feeling like a summary of Wikipedia.

Chance-Device• 1 year ago

And the AI stocks fell today.

I’m sure it’s unrelated.

boznz• 1 year ago

So better than 4o but not good enough for a 5.0

simonw4• 1 year ago

I got gpt-4.5-preview to summarize this discussion thread so far (at 324 comments):

  hn-summary.sh 43197872 -m gpt-4.5-preview

Using this script: https://til.simonwillison.net/llms/claude-hacker-news-themes...

Here's the result: https://gist.github.com/simonw/5e9f5e94ac8840f698c280293d399...

It took 25797 input tokens and 1225 input tokens, for a total cost (calculated using https://tools.simonwillison.net/llm-prices ) of $2.11! It took 154 seconds to generate.

djhworld• 1 year ago

interesting summary but it's hard to gauge whether this is better/worse than just piping the contents into a much cheaper model.

azinman22• 1 year ago

It’d be great if someone would do that with the same data and prompt to other models.

I did like the formatting and attributions but didn’t necessarily want attributions like that for every section. I’m also not sure if it’s fully matching what I’m seeing in the thread but maybe the data I’m seeing is just newer.

joe_the_user2• 1 year ago

The headline and section: "Dystopian and Social Concerns about AI Features" are interesting. It's roughly true... but somehow that broad statement seems minimize the point discussed.

I'd headline that thread as "Concerns about output tone". There were comments about dystopian implications of tone, marketing implications of tone and implementation issues of tone.

Of course, that I can comment about the fine-points of an AI summary shows it's made progress. But there's a lot riding on how much progress these things can make and what sort. So it's still worth looking at.

alew1• 1 year ago

Didn't seem to realize that "Still more coherent than the OpenAI lineup" wouldn't make sense out of context. (The actual comment quoted there is responding to someone who says they'd name their models Foo, Bar, Baz.)

willy_k3• 1 year ago

Wonder if there’s some pro-OpenAI system prompt getting in the way of that.

colordrops• 1 year ago

Maybe it's just confirmation bias but the language in your result output seems higher quality that previous models. Seems more natural and eloquent.

3vidence• 1 year ago

I don't know why but something about this section made me chuckle

""" These perspectives highlight that there remains nuance—even appreciation—of explorative model advancement not solely focused on immediate commercial viability """

Feels like the model is seeking validation

stevage3• 1 year ago

Huh. Disregarding the 4.5-specific bit here, a browser extension or possibly website that did this in general could be really useful.

Maybe even something that just noticed whenever you visited a site that had had significant HN discussion in the past, then let you trigger a summary.

wordpad252• 1 year ago

there are literally hundreds of extensions and sites that do this

the problem is that they are competing each other into the ground hence they go unmaintained very quickly

getrecall.ai has been the most mature so far

ukuina• 1 year ago

My site https://hackyournews.com does this!

Been keeping it alive and free for 18 months.

sebastiennight16• 1 year ago

What I want is something that can read the thread out loud to me, using a different voice per user, so I can listen to a busy discussion thread like I would listen to a podcast.

dahsameer• 1 year ago

$2.11! At this point, I'm more concerned about AI price than egg price.

vivzkestrel• 1 year ago

you know what? it would be damn nice to do this to literally every post in HN and give people a summary so that they dont have to read 500 comments

munksbeer• 1 year ago

As expected, comments on LLM threads are overwhelmingly negative.

Personally, I still feel excited to see boundaries being pushed, however incremental our anecdotal opinions make them seem.

sundarurfriend• 1 year ago

I disagree with most of the knee-jerk negativity in LLM threads, but in this case it mostly seems warranted. There are no "boundaries being pushed" here, this is just a desperate release from a company that finds itself losing more and more mindshare to other models and companies.

egobrain27• 1 year ago

Seems to have trouble recognizing sarcasm:

"For example, there are now a bunch of vendors that sell 'respond to RFP' AI products... paying 30x for marginally better performance makes perfect sense." — hn_throwaway_99 (an uncommon opinion supporting possible niche high-cost uses).

mlyle• 1 year ago

? You think hn_throwaway_99's comment is sarcastic? It makes perfect sense to me read "straight."

That is, sales orgs save a bunch of money using AI to respond to RFPs; they would still save a bunch of money using a more expensive AI, and any marginal improvement in sales closed would pay for it.

It maybe excessively summarized his comment which confused you-- but this is the kind of mistake human curators of quotes make, too.

sebzim45003• 1 year ago

I don't think they are being sarcastic. Maybe you are the bot /s

orbital-decay2• 1 year ago

This looks like a first generation model to bootstrap future models from, not a competitive product at all. The knowledge cutoff is pretty old as well. (2023, seriously?)

If they wanted to train it to have some character like Anthropic did with Claude 3... honestly I'm not seeing it, at least not in this iteration. Claude 3 was/is much much more engaging.

GaggiX• 1 year ago

I imagine it will be used as a base for GPT-5 when it will be trained into a reasoning model, right now it probably doesn't make too much sense to use.

dgfitz4• 1 year ago

@sama, LLMs aren't going to create AGI. I realize you need to generate cash flow, this isn't the play.

Sincerely, Me

kneegerman• 1 year ago

you misunderstand, the business model is extracting cash from Qatar et al

vhiremath4• 1 year ago

I cancelled my ChatGPT subscription today in favor of using Grok. It’s literally the difference between me never using ChatGPT to using Grok all the time, and the only way I can explain it is twofold:

1. The output from Grok doesn’t feel constrained. I don’t know how much of this is the marketing pitch of it “not being woke”, but I feel it in its answers. It never tells me it’s not going to return a result or sugarcoats some analysis it found from Reddit that’s less than savory.

2. Speed. Jesus Christ ChatGPT has gotten so slow.

Can’t wait to pay for Grok. Can’t believe I’m here. I’m usually a big proponent of just sticking with the thing that’s the most popular when it comes to technology, but that’s not panning out this time around.

tiahura2• 1 year ago

I found Grok's reasoning pretty wack.

I asked it - "Draft a Minnesota Motion in Limine to exclude ..."

It then starts thinking ... User wants a Missouri Motion in Limine ....

blitq• 1 year ago

It's just nuts how pricy this model is when scoring worse than o3-mini

torginus4• 1 year ago

Call me a conspiracy theorist, but this, combined with the extremely embarassing way Claude is playing Pokemon, makes me feel this is an effort by AI companies to make LLMs look bad - setting up the hype cycle for the next thing they have in the pipeline.

camdenreslink3• 1 year ago

The next thing in the pipeline is definitely agents, and making the underlying tech look bad won't help sell that at all.

torginus4• 1 year ago

Agents as they are right now is literally just the LLM calling itself in a loop + having the ability to use tools/interact with their environment. I don't know if there's anything profoundly disruptive cooking in that space.

afastow2• 1 year ago

You're not a conspiracy theorist, you're just recognizing that the reality doesn't match the hype. It's boring and not fun but in this situation the answer is almost always that the hype is wrong, not the reality.

sunami-ai• 1 year ago

still can't deal with sequences (or permutations)

https://chatgpt.com/share/67c0f064-7fdc-8002-b12a-b62188f507...

The Share doesn't say 4.5 but I assure you it is 4.5

adamtaylor_13• 1 year ago

It’s crazy how quickly OpenAI releases went from, “Honey, check out the latest release!” to a total snooze fest.

Coming in the heels of Sonnet 3.7 which is a marked improvement over 3.5 which is already the best in the industry for coding, this just feels like a sad whimper.

404mm• 1 year ago

I’m just disappointed that while everyone else (DS, Claude) had something to introduce for the “Plus” grade users, gpt 4.5 is so resource demanding that it’s only available to quite expensive Pro sub. That just doesn’t feel much like progress.

gcanyon• 1 year ago

The prices they’re charging are not that far from where you could outsource to a human.

resters• 1 year ago

based on a few initial tests GPT-4.5 is abysmal. I find the prose more sterile than previous models and far from having the spark of DeepSeek, and it utterly choked on / mangled some python code (~200 LoC and 120 LoC tests) that o3-mini-high and grok-3 do very well on.

xena• 1 year ago

I'm really not sure who this model is for. Sure the vibes may be better, but are they 2.5x as much as o1 better? Kinda feels like they're brute forcing something in the backend with more hardware because they hit a scaling wall.

yousif_123123• 1 year ago

It's disappointing not to see comparisons to Sonnet 3.7. Also since o3-mini is ahead of o1, not sure why in the video they compared to o1.

gpt4 was way ahead of 3.5 when it came out. It's unfortunate that the first major gpt release since that is so underwhelming..

j_bum4• 1 year ago

Agreed, but I suppose this is a tell. I think they’re trying to place this into a separate class of models.

I.e., we know it might not be as good as 3.7, but it is very friendly and maybe acts like it knows more things.

IAmGraydon• 1 year ago

Between this and Claude 3.7, I'm really beginning to believe that LLM development has hit a wall, and it might actually be impossible to push much farther for reasonable amounts of money and resources. They're incredible tools indeed and I use them on a daily basis to multiply my productivity, but yeah - I think we've all overshot this in a big way.

j_bum4• 1 year ago

Agreed at every level.

I absolutely love LLMs. I see them as insanely useful, interactive, quirky, yet lossy modern search engines. But they’re fundamentally flawed, and I don’t see how an “agent” in the traditional sense of the world can actually be produced from them.

The wall seems to be close. And the bubble is starting to leak air.

netdevphoenix2• 1 year ago

LLM development has hit a wall

The writing has been on the wall since 2024. None of the LLM releases have been groundbreaking they have all been lateral improvements and I believe the trend will continue this year with make them more efficient (like DeepSeek), make them faster or make them hallucinate less

passwordoops• 1 year ago

Still no 5, huh?

andrewinardeer• 1 year ago

I just played with the preview through the API. I asked it to refactor a fairly simple dashboard made with HTML, css and JavaScript.

First time it confused css and JavaScript, then spat out code which broke the dashboard entirely.

Then it charged me $1.53 for the privilege.

ipnon• 1 year ago

Finally a replacement for junior engineers!

pronouncedjerry• 1 year ago

instead of these random IDs they should label them to make sense for the end user. i have no idea which one to select for what i need. and do they really differ that much by use case?

sirolimus• 1 year ago

Slow, expensive and nothing special. Just stick to o1 or give us o3 (non-mini).

msp26• 1 year ago

"GPT4.5, the most knowledgable model to date" Knowledge cutoff: October 2023

bag_boy• 1 year ago

I tried it and if it’s more natural, I don’t know what that means anymore, because I’m used to the last 6 month’s models.

mrcwinn3• 1 year ago

Sam continues to be the least impressive person to ever lead such an amazing company.

Odd communication from him recently too. We’re sorry our UI has become so poor. We’re sorry this model is so expensive and not a big leap.

Being rich and at the right place at the right time doesn’t itself qualify you to lead or make you a visionary. Very odd indeed.

tuananh• 1 year ago

this seems to be a very weak response to sonnet 3.7

- more expensive. alot more expensive

- not a lot of increment improvement

energy1235• 1 year ago

Lower hallucinations than o1. Impressive.

hombre_fatal3• 1 year ago

Cathartic moment over.

apwell233• 1 year ago

doesn't feel like to me. I try using copilot on my scala projects and it always comes up with something useless that doesn't even compile.

I am currently just using it as easy google search.

abrichr• 1 year ago

Have you tried copying the compilation errors back into the prompt? In my experience eventually the result is correct. If not then I shrink the surface area that the model is touching and try again.

NiloCK• 1 year ago

Breath it in, get a coffee, and sit down to solve some bigger problems.

3.7 really is astounding with the one-shots.

tmpz223• 1 year ago

I haven't had the same experience. Here are some of the significant issues when using o1 or claude 3.7 with vscode copilot:

* Very wreckless in pulling in third party libraries - often pulling in older versions including packages that trigger vulnerability warnings in package managers like npm. Imagine a student or junior developer falling into this trap.

* Very wreckless around data security. For example in an established project it re-configured sqlite3 (python lib) to disable checks for concurrent write liabilities in sqlite. This would corrupt data in a variety of scenarios.

* It sometimes is very slow to apply minor edits, taking about 2 - 5 minutes to output its changes. I've noticed when it takes this long it also usually breaks the file in subtle ways, including attaching random characters to a string literal which I very much did not want to change.

* Very bad when working with concurrency. While this is a hard thing in general, introducing subtle concurrency bugs into a codebase is not good.

* By far is the false sense of security it gives you. Its close enough to being right that a constant incentive exists to just yeet the code completions without diligent review. This is really really concerning as many organizations will yeet this, as I imagine executives are currently the world over.

Honestly I think a lot of people are captured by a small sample size of initial impressions, and while I believe you in that you've found value for use cases - in aggregate I think it is a honeymoon phase that wears off with every-day use.

hombre_fatal3• 1 year ago

I've been using it daily for years. Mostly asking questions in a separate chat window/app and then working its response into my code. And then I sped up the feedback loop when I migrated to Cursor where I began pushing the envelop and asking it to do more.

I think what wears off is that we're less impressed and then we start demanding more and more from it and getting frustrating when it can't do it. But that's different than a honeymoon phase wearing off. It's like how we're not really impressed by image gen anymore, we expect it.

But as an example of a selfish sense of loss I've experienced, I used to pride myself in being the only developer on any team who ever learned CSS. I could architect a good grid/flex layout with a lot of thought. I could do little things like make text in a small UI component truncate into {3 letters} + ellipses when its parent was too small. And most of all I could polish UIs to a point where I'd say they were perfect, even a form.

Now, LLMs are really good at doing the mechanical parts of the things I spent so much time learning. Like I originally said, I'm not shedding tears over here saying it's so unfair. But there is a sense of loss. And when I figured most people reading my comment would misinterpret this, I removed my comment. Because you can't make descriptive claims about how you feel online, it can only be interpreted as a normative value judgement about the world. Because I guess that's what it is 99.9% of the time someone expresses a feeling they feel, but not in this case.

Finally, the right way to see it is that now I can polish the UI to perfection, but I don't need to be a CSS expert anymore. Nobody needs to be. You can get an idea of how you want the UI to work and ask the LLM "make this one bit of text be the one that truncates if the window is too narrow" and it does it. And that's fkin magic.

apwell233• 1 year ago

why did you remove the comment . now who ppl responded to you look like dummies. do you do this sort of stuff in real life too?

hombre_fatal3• 1 year ago

What would it mean to do this in real life? :D

I regularly make knee-jerk comments on HN that I delete a minute later. Something therapeutic about it.

My comment isn't one I wanted on my "record". You responded to it and I saw your response before deleting my comment. What's the harm? It's obvious I removed my comment.

fungiblecog• 1 year ago

Enjoy your expensive garbage

nycdatasci3• 1 year ago

When I asked “what version are you?” it insisted that it was ChatGPT 4.0 Turbo, one step behind GPT-4.5. https://chatgpt.com/share/67c0fda8-a940-800f-bbdc-6674a8375f...

xeckr• 1 year ago

My initial impression is that I have gotten quite spoiled by the speed of GPT-4o...

dangoodmanUT• 1 year ago

Finally, an LLM that doesn't YAP

prompt_judy• 1 year ago

Its not even that great for real world business tasks. I have no idea what they are thinking https://youtu.be/puPybx8N82w

ahmadtbk• 1 year ago

Will this pop the AI bubble?

camdenreslink3• 1 year ago

I think if GPT-5 is very underwhelming we could start to see some shifting of opinion on what kind of return on investment all of this will result in.

afastow2• 1 year ago

This is GPT-5, or rather what they clearly intended to be GPT-5. The pricing makes it obvious that the model is massive, but what they ended up with wasn't good enough to justify calling it more than 4.5.

stan_kirdey• 1 year ago

For most tasks, GPT-4o/o3-mini are already great, and cheaper.

What is the real-world use case where GPT-4.5? Anyone actually seeing a game-changing difference? Please share your prompts!

gigagorilla• 1 year ago

Bring back GPT-1. It really knew how to have a conversation.

gorgoiler• 1 year ago

I love the “listen to this article” widget doing embedded TTS for the article. Bugs / feedback:

The first words I hear are “introducing gee pee four five”. The TTS model starts cold? The next occurrence of the product name works properly as “gee pee tee four point five” but that first one in the title is mangled. Some kind of custom dictionary would help here too, for when your model needs to nail crucial phrases like your business name and your product.

No way of seeking back and forth (Safari, iOS 17.6.1). I don’t even need to seek, just replay the last 15s.

Very much need to select different voice models. Chirpy “All new Modern Family coming up 8/9c!” voice just doesn’t cut it for a science broadcast, and localizing models — even if it’s still English — would be even better. I need to hear this announcement in Bret Taylor voice, not Groupon CMO voice. (Sorry if this is your voice btw, and you work at OpenAI brandi. No offence intended.)

Perenti• 1 year ago

What I find hilarious is that a 20-50% hallucination rate suggests this is still a program that tells lies and potentially causes people to die.

oblio4• 1 year ago

program that tells lies and potentially causes people to die.

Sounds like a valid legal defense for crummy companies.

iamronaldo• 1 year ago

https://livebench.ai/#/ Best non reasoning model on livebench (and ranks above gemeni thinking)

anshumankmr4• 1 year ago

If this cannot eliminate hallucinations or at least reduce them to be statistically unlikely to be happen, and I assume it has more params than GPT4's trillion parameters, that means the scaling law is dead isn't it?

energy1235• 1 year ago

I interpret this to mean we're in the ugly part of the old scaling law, where `ln(x)` for `x > $BIGNUMBER` starts to becoming punishing, not that the scaling law is in any way empirically refuted. Maybe someone can crunch the numbers and figure out if the benchmarks empirically validate the scaling law or not, relative to GPT-4o (assuming e.g. 200 million params vs 5T params).

sudosysgen2• 1 year ago

I mean the scaling laws were always logarithms, and logarithms become arbitrarily close to flat if you can't drive them with exponential growth, and even if you do it's barely linear. The scaling laws always predicted that model scaling would stop/slow being practical at some point.

anshumankmr4• 1 year ago

Right but the quantum leap in capabilities that came from GPT2->GPT3->GPT3.5Turbo (which I personally felt didn't fare as well at coding as the former)->GPT4 won't be replicated anytime soon with the pure text/chat generation models.

ksynwa2• 1 year ago

Why would they want to eliminate that? Altman said that hallucinations are how LLMs express creativity.

mrcwinn3• 1 year ago

First time I’ve had an LLM reply “Nope.”

https://chatgpt.com/share/67c154e7-5e28-800d-81d7-98b79c8a87...

loa_observer• 1 year ago

You got 10x price but not 10x quality

mrcwinn3• 1 year ago

I’d prefer this model if it were faster, but not at this cost. And so it is an odd release.

Still, with Deep Research and Web Search, ChatGpt seems far ahead of Claude. I like 3.7 a lot but I find OpenAI’s features more useful, even if it has for now complicated the UI a bit.

fsloth2• 1 year ago

Agree on the Web App. Cursor with Claude 3.7 is a pretty good "CoPilot" experience though.

cubefox• 1 year ago

Altman mentioned GPT-4.5 is the model code named "Orion". Which originally was supposed to be their next big model, presumably GPT-5, but showed disappointing improvements on benchmark performance. Apparently the AI companies are hitting diminishing returns with the paradigm of scaling foundation model pretraining. It was discussed a few months ago:

https://news.ycombinator.com/item?id=42125888

DennisL123• 1 year ago

tl;dr: doesn‘t work as expected and we sank a ton of money on it too.

yobid20• 1 year ago

Cash grab because they see the writing on the wall. OpenAI is collapsing. Their models suck now.

weinzierl• 1 year ago

"Starting today, ChatGPT Pro users will be able to select GPT‑4.5 in the model picker on web, mobile, and desktop. We will begin rolling out to Plus and Team users next week, then to Enterprise and Edu users the following week."

Thanks for being transparent about this. Nothing is more frustrating than being locked out for indeterminate time from the hot thing everyone talks about.

I hope the announcement is true without further unsaid qualifications, like availability outside the US.

vbezhenar4• 1 year ago

I'm outside the US and I have access to ChatGPT 4.5 with ChatGPT Pro subscription. Didn't have that access yesterday at the time of announce, but probably they were staggering the release a bit to even the load over multiple hours.

yzydserd• 1 year ago

Sadly, it has a small context window.

nbzso• 1 year ago

Yesterday I tested Windsurf. Looked the docs and examples. Completed the demo "course" on deeplearning.ai. Gave it the task to build a simple Hugo blog website with a theme link and requirements, it failed consecutive times. With all the available models.

AI art is an abomination. Half of the internet is already filled with AI written crap. Don't start with the video. Soon everyone will require validation to distinguish reality from hallucination (so World ID in place as problem-reaction-solution).

For me, the best use cases are LLM assisted search with limited reasoning. Vision models for digitization and limited code assistance, codebase doc generation and documentation.

Agents are just workflows with more privileges. So where is the revolution? I don't see it.

Where is added value? Making Junior Engineers obsolete? Or make them dumb copy-pasting bio machines?

Depressing a horde of intellectual workers and artists and giving a good excuse for layoffs.

The real value is and always will be in a specialized ML applications.

LLM hype is getting boring.

ta3456345345• 1 year ago

The AI hyperbole is so cringe right now (and for the last few years). I've yet to see anyone come up with something that'd wow me, and say, "OK, yep, that deserves those cycles".

Writing terrible fanfic esque books, sometimes OK images, chatbot style talking. meh.

PeterStuer2• 1 year ago

Currently my daily API costs for 4o are low enough and performance/quality for my usecases good enough that switching models has not made to to the top of application improvements.

My cases' costs are more heavily slanted towards input tokens, so trying 4.5 would raise my costs over 25x, which is a non-starter.

jsemrau• 1 year ago

Interesting observation.It seems capability has reached a plateau. Like a local maximum.

PeterStuer2• 1 year ago

I'm not sure that is the right conclusion.

It is more like the AI part of the system for this specific use case has reached a position where focusing on that part of the complete application as opposed to other parts that need attention would not yield the highest return in terms of user satisfaction or revenue.

Certainly there is enormous potential for AI improvement, and I have other projects that do gain substantially from improvements in e.g. reasoning, but then GPT 4.5 will have to compete with Deepseek, Gemini, Grok and Claude on a price/performance level, but to be honest the current preview pricing would make it (in production, not for dev) a non starter for me.

aubanel2• 1 year ago

The price is not that insane when you remember that GPT-4 cost 36$/million input tokens at launch!

Tepix• 1 year ago

Prices have come down

aubanel2• 1 year ago

yes, which suggests they'll keep going down! So I'd expect GPT-4.5 to be 90% cheaper in 1-2 years

netdevphoenix2• 1 year ago

Is it official then?

Most of us have been waiting for this moment for a while. The transformer architecture as it is currently understood can't be milked any further. Many of us knew this since last year. GPT-5 delays eventually led to non-tech voices to suggest likewise. But we all held our final decision until the next big release from OpenAI as Sam Altman has been making claims about AGI entering the workforce this year, OpenAI knowing how to build AGI and similar outlandish claims. We all knew that their next big release in 2025 would be the final deciding factor on whether they had some tech breakthrough that would upend the world (justifying their astronomical valuation) or if it would just be (slightly) more of the same (marking the beginning of their downfall).

The GPT-4.5 release points towards the latter. Thus, we should not expect OpenAI to exist as it does now (AI industry leader) in 2030, assuming it does exist at all by then.

However, just like the 19th century rail industry revolution, the fall of OpenAI will leave behind a very useful technology that while not catapulting humanity towards a singularity, will nonetheless make people's lives better. Not much consolation to the world's super rich who will lose tons of money once the LLM industry (let us remember that AI is not LLM) falls.

EDIT: "will nonetheless make people's lives better" to "might nonetheless make some people's lives better"

fergonco• 1 year ago

will nonetheless make people's lives better

Probably not the lives of translators or graphic designers or music compositors. They will have to find new jobs. As llm prompt engineers, I guess.

yurishimo• 1 year ago

Graphic designers I think are safe, at least within organizations that require a cohesive brand strategy. Getting the AI to respect all of the previous art will be a challenge at a certain scale.

Fiverr graphic designers on the other hand…

entropi• 1 year ago

will nonetheless make people's lives better

While I mostly agree with your assessment, I am still not convinced of this part. Right now, it may be making our lives marginally better. But once the enshittification starts to set in, I think it has the potential to make things a lot worse.

E.g. I think the advertisement industry will just love the idea of product placements and whatnots into the AI assistant conversations.

dkdcwashere• 1 year ago

*good*. the answer to this is legislation —- legally, stop allowing shitty ads everywhere all the time. I hope these problems we already have are exacerbated by the ease of generating content with LLMs and people actually have to think for themselves again

km1442• 1 year ago

I'm not convinced that LLMs in their current state are really making anyone's lives much better though. We really need more research applications for this technology for that to become apparent. Polluting the internet with regurgitated garbage produced by a chat bot does not benefit the world. Increasing the productivity of software developers does not help to the world. Solving more important problems should be the priority for this type of AI research & development.

dgsm98• 1 year ago

Solving more important problems should be the priority for this type of AI research & development.

Which problem spaces do you think are underserved in this aspect?

pera• 1 year ago

The explosion of garbage content is a big issue and has radically changed the way I use the web over the past year: Google and DuckDuckGo are not my primary tools anymore, instead I am now using specialized search engines more and more, for example, if I am looking for something I believe can be found in someone's personal blog I just use Marginalia or Mojeek, if I am searching for software issues I use GitHub's search, general info straight to Wikipedia, tech reviews HN's Algolia etc.

It might sound a bit cumbersome but it's actually super easy if you assign search keywords in your browser: for instance if I am looking for something on GitHub I just open a new tab on Firefox and type "gh tokio".

Workaccount23• 1 year ago

LLM's have been extremely useful for me. They are incredibly powerful programmers, from the perspective of people who aren't programmers.

Just this past week claude 3.7 wrote a program for us to use to quickly modernize ancient (1990's) proprietary manufacturing machine files to contemporary automation files.

This allowed us to forgo a $1k/yr/user proprietary software package that would be able to do the same. The program Claude wrote took about 30 mins to make. Granted the program is extremely narrow in scope, but it does the one thing we need it to do.

This marks the third time I (a non-progammer) have used an LLM to create software that my company uses daily. The other two are a test system made by GPT-4 and an android app made by a mix of 4o and claude 3.5.

Bumpers may be useless and laughable to pro bowlers, but a godsend to those who don't really know what they are doing. We don't need to hire a bowler to knock over pins anymore.

rjinman4• 1 year ago

As someone who is terrified of agentic ASI, I desperately hope this is true. We need more time to figure out alignment.

cle• 1 year ago

I'm not sure this will ever be solved. It requires both a technical solution and social consensus. I don't see consensus on "alignment" happening any time soon. I think it'll boil down to "aligned with the goals of the nation-state", and lots of nation states have incompatible goals.

rgbrenner• 1 year ago

"alignment" is a bs term made up to deflect blame from the overpromises the AI companies made to hype up their product to obtain their valuations.

drdaeman• 1 year ago

It doesn't do anyone any good to stress over non-existent things. ASI is a sci-fi trope, a pure fantasy in context of present day and time. AGI does not exist either, and AFAIK there's not even any agreement what it possibly means beyond very vague "no worse than a human".

In other words, I'm sure you're terrified of a modern fairy tale.

aurareturn3• 1 year ago

Honestly, I'm not sure how you can make all those claims when:

1. OpenAI still has the most capable model in o3

2. We've seen some huge increases in capability in 2024, some shocking

3. We're only 3 months into 2025

4. Blackwell hasn't been used to train a model yet

PaulRobinson• 1 year ago

It's worth pointing out that GPT-4.5 seems focused on better pre-training and doesn't include reasoning.

I think GPT-5 - if/when it happens - will be 4.5 with reasoning, and as such it will feel very different.

The barrier, is the computational cost of it. Once 4.5 gets down to similar costs to 4.0 - which could be achieved through various optimization steps (what happened to the ternary stuff that was published last year that meant you could go many times faster without expensive GPUs?), and better/cheaper/more efficient hardware, you can throw reasoning into the mix and suddenly have a major step up in capability.

I am a user, not a researcher of builder. I do think we're in a hype bubble, I do think that LLMs are not The Answer, but I also think there is more mileage left in this path than you seem to. I think automated RL (not HF), reasoning, and better/optimal architectures and hardware mean there is a lot more we can get out of the stochastic parrots, yet.

highfrequency2• 1 year ago

Is it fair to still call LLMs stochastic parrots now that they are enriched with reasoning? Seems to me that the simple procedure of large-scale sampling + filtering makes it immediately plausible to get something better than the training distribution out of the LLM. In that sense the parrot metaphor seems suddenly wrong.

I don’t feel like this binary shift is adequately accounted for among the LLM cynics.

glenstein2• 1 year ago

the barrier, is the computational cost of it. Once 4.5 gets down to similar costs to 4.0

Well, did 4.0 ever become lower cost? On the API side, its cost per tokens is a factor of 10 higher than 4o even though 4o is considered the better model.

I think 4.5 may just be retired wholesale, or perhaps a new model derived from it that is more efficient, a 4.5mini or something like that.

vbezhenar4• 1 year ago

I feel like it was GPT-5 which was eventually renamed to keep up with expectations.

NotYourLawyer• 1 year ago

Not much consolation to the world's super rich who will lose tons of money once the LLM industry (let us remember that AI is not LLM) falls.

They knew the deal:

“it would be wise to view any investment in OpenAI Global, LLC in the spirit of a donation” and “it may be difficult to know what role money will play in a post-[artificial general intelligence] world.”

qoez• 1 year ago

It's always been a combination of data and scale (garbage data on massive scale gives garbage still). Data is continually getting better though so we'll still be able to squeeze a lot out of transformers yet

sebzim45003• 1 year ago

This seems very dramatic given OpenAI still has the best model in the world `o3`.

oblio4• 1 year ago

The best model in the world is still basically a very stubborn, yet mediocre 16 year old with a memory the size of the internet.

diego_sandoval• 1 year ago

OpenAI knowing how to build AGI and similar outlandish claims.

The fact that the scaling of pretrained models is hitting a wall doesn't invalidate any of those claims. Everyone in the industry is now shifting towards reasoning models (a.k.a. chain of thought, a.k.a. inference time reasoning, etc.) because it keeps scaling further than pretraining.

Sam said the phrase you refer to [1] in January, when OpenAI had already released o1 and was preparing to release o3.

[1] https://blog.samaltman.com/reflections

mountainriver• 1 year ago

lol this isn’t a reasoning model, those are doing very well, but cute essay you wrote there

energy1235• 1 year ago

~40% hallucinations on SimpleQA by a frontier reasoner (o1) and a frontier non-reasoner (GPT-4.5). More orders of magnitude in scale isn't going to fix this deficit. There's something fundamentally wrong with the approach. A human is much more capable of saying "I don't know" in the correct spots, even if a human is also susceptible to false memories.

Probably OpenAI thinks that tool use (search) will be sufficient to solve this problem. Maybe that will be the case.

Are there any creative approaches to fixing this problem?

amelius2• 1 year ago

With every new model I'd like to see some examples of conversations where the old model performed badly and the new model fixes it. And, perhaps more importantly, I'd like to see some examples where the new model can still be improved.

theshrike79• 1 year ago

I want less and less of these "do it all models", what I want is specific models for the exact task I need.

Then what I want is a platform with a generic AI on top that can pick the correct expert models based on what I asked it to do.

Kinda what Apple is attempting with their Small Language Model thing?

DrNosferatu• 1 year ago

Is it available on the web client?

darthrupert• 1 year ago

Here's a reminder that an European alternative exists: https://chat.mistral.ai

It lags somewhat behind the US offerings, but a great way to fix that is to direct some money at them. Also, as things progress, that lagging starts being less and less important.

Curzel• 1 year ago

I have no idea why they bothered releasing it, but, as we saw with the other models, they will eventually release a 4.5-turbo (or a 4.5o) of sorts that is in line with current pricing

amelius2• 1 year ago

How does it compare against IBM's Watson?

overflyer• 1 year ago

They bring out 4.5 while all functions of 4o,so not so omni heh, are still not available.

DonHopkins5• 1 year ago

The latest claude-3.7-sonnet-thinking is fucking amazing with Cursor. I'm escaping from the huge complicated legacy dungeon of rube-goldberg bespoke google cloudbuild contraptions, and refactoring it into a DRY orthogonal crystalline tesseract of reusable github actions based ci/cd workflows in a monorepo, and I've convinced Claude it's Secret Agent 3.7 and I'm his handler.

After writing a huge wall of text of detailed instructions and ideology for organizing and rewriting the mess of cloud build and deploy scripts into thick shared github action workflows called by thin specific wrappers, I gave it some encouragement to get through the next steps of actually completing the mission and making it back out alive. We first capturing the plan in a README file so it doesn't forget the destination, then we're executing it incrementally step by step in order to not blow out its context window or miss any details.

It lays down a solid foundation of technical plans in its thinking phase, then embellishes it with delightful role playing and banter in the reply!

Programming is really fun again!

User:

I feel like a secret agent handler in James Bond or the Matrix telling you which hall to turn down and door to open and remotely unlocking them and disabling burgler alarms! Turn right and go through the door at the end of the hall 007!

Claude:

<think>

You're expressing pleasure at our coordinated teamwork - you're directing which files to modify and I'm executing the changes with precision. Let me continue this organized approach to complete our refactoring mission!

Looking at our current status:

We've updated the README with the new structure information

We've created the shared workflow files (shared-docker-build.yml and shared-docker-deploy.yml)

We've started updating service-specific workflows to use the shared ones

Next, I should examine what changes are needed for the pyvision-deploy.yml to use our new shared-docker-deploy.yml workflow.

</think>

Mission Status Update: Agent 3.7 Reporting

[Adjusts earpiece] Copy that, Control. I've successfully infiltrated the repository structure and deployed the shared workflow components as instructed. The orchestration pattern is now in place.

[Checks surroundings] I've identified our next target: pyvision-deploy.yml needs to be refactored to utilize the new shared-docker-deploy.yml workflow we've established.

Next Objective:

Looking at the file structure, we need to:

Update pyvision-deploy.yml to use our shared deployment workflow

Fix the disconnected workflow reference (deploy-pyvision.yml vs pyvision-deploy.yml)

Update the path references in the push triggers

[Checks watch] Should I proceed with these modifications immediately, or would you like me to scout ahead for any potential complications?

[Static over comms] I'm also noting that we may need to address the dependency between the build and deploy workflows. The current pyvision-build.yml is attempting to trigger deploy-pyvision.yml, but our standardized pattern uses pyvision-deploy.yml.

What are your instructions, Control?

User:

Give me a concise statement of our organizational ideology, like the Zen of Python.

Claude:

  The Zen of CI/CD:

  Consistency over convenience.
  Explicitness beats implicitness.
  Simple naming conventions create self-documenting systems.
  Big-endian naming sorts better than little-endian.
  Alphabetical order reveals what belongs together.
  Shared implementation reduces cognitive load.
  Technology, component, action, and abstraction are our four dimensions.
  Tesseracts are better organized than flat directories.
  Special cases aren't special enough to break the rules.
  Except when they're waiting to be refactored.
  Workflows should be orthogonal, reusable, and composable.
  In the face of ambiguity, refuse the temptation to guess.

User:

Brilliant! Please add that to the README in the beginning as a manifesto.

...Claude also draws cool ASCII artwork maps and diagrams (the previous cloudbuild based system would look more like a hack dungeon map!):

  ### Workflow Diagram

  ```
         SHARED WORKFLOWS                 SPECIFIC SERVICES
  ┌───────────────────────────┐      ┌──────────────────────────┐
  │                           │      │                          │
  │  ┌─────────────────────┐  │      │  ┌──────────┐ ┌────────┐ │
  │  │ shared-docker-build │◄─┼──────┼──┤ pyvision-│ │concept-│ │
  │  └──────────┬──────────┘  │      │  │  build   │ │ build  │ │
  │             │             │      │  └────┬─────┘ └───┬────┘ │
  │             ▼             │      │       │           │      │
  │  ┌─────────────────────┐  │      │  ┌────▼─────┐ ┌───▼────┐ │
  │  │ shared-docker-deploy│◄─┼──────┼──┤ pyvision-│ │concept-│ │
  │  └─────────────────────┘  │      │  │  deploy  │ │ deploy │ │
  │                           │      │  └──────────┘ └────────┘ │
  │  ┌─────────────────────┐  │      │                          │
  │  │ shared-worker-build │◄─┼──────┼──┐                       │
  │  └──────────┬──────────┘  │      │  │                       │
  │             │             │      │  │                       │
  │             ▼             │      │  │  ┌──────────┐         │
  │  ┌─────────────────────┐  │      │  └──┤  looker- │         │
  │  │ shared-worker-deploy│◄─┼──────┼─────┤   build  │         │
  │  └─────────────────────┘  │      │     └────┬─────┘         │
  │                           │      │          │               │
  │                           │      │     ┌────▼─────┐         │
  │                           │      │     │  looker- │         │
  │                           │      │     │  deploy  │         │
  │                           │      │     └──────────┘         │
  └───────────────────────────┘      └──────────────────────────┘
                 │                                               
                 ▼                                               
         ┌──────────────────┐                                    
         │ Script Utilities │                                    
         └──────────────────┘                                    
  ```

fragmede6• 1 year ago

That's amazing!

FailMore• 1 year ago

Funny times. Sonnet 3.7 launches and there is big hype... but complaints start to surface on r/cursor that it is doing too much, is too confident, has no personality. I wonder if 4.5 will be the reverse, an under-hyped launch, but a dawning realisation that it is incredibly useful. Time will tell!

Etheryte2• 1 year ago

I share the sentiment, as far as I've used it, Sonnet 3.7 is a downgrade and I use Sonnet 3.5 instead. 3.7 tends to overlook critical parts of the query and confidently answers with irrelevant garbage. I'm not sure how QA is done on LLM-s, but I for one definitely feel like the ball was dropped somewhere.

anti-soyboy4• 1 year ago

Open AIscam

A_D_E_P_T2• 1 year ago

I've been using 4.5 for the better part of the day.

I also have access to o3-mini-high and o1-pro.

I don't get it. For general purposes and for writing, 4.5 is no better than o3-mini. It may even be worse.

I'd go so far as to say that Deepseek is actually better than 4.5 for most general purpose use cases.

I seriously don't understand what they're trying to achieve with this release.

leumon• 1 year ago

this model does have a niche use-case: since its so large it does have a lot more knowledge and hallucinates much less. for example as a test question I asked it to list the best restaurants in my small town. and all of them existed. none of the other llms get this right.

A_D_E_P_T2• 1 year ago

I tried the same thing with companies in my industry ("list active companies in the field of X") and it came back with a few that have been shuttered for years, in one case for nearly two decades.

I'm really not seeing better performance than with o3-mini.

If anything, the new results ("list active companies in the field of X") are actually worse than what I'd get with o3-mini, because the 4.5 response is basically the post-SEO Google first page (it appears to default to mentioning the companies that rank most highly on Google,) whereas the o3 response was more insightful and well-reasoned.

lolinder2• 1 year ago

That's also a use case where the consensus among those in the know is that you shouldn't be relying on the model's size in the first place.

You know what gets the list of restaurants in my home town right? Llama 3.2 1b q4 running on my desktop with web search enabled.

roody15• 1 year ago

Interesting times that are changing quickly. It looks like the high end pay model that OpenAI is implementing may not be sustainable. Too many new players are making LLM breakthroughs and OpenAI's lead is shrinking and it may be overvalued.

devmor• 1 year ago

A 30x price increase with zero named benefits?

This sure looks like the runway is about to come far short of takeoff. I’m reminded of Ed Zitron’s recent predictions…

brador• 1 year ago

There are companies that will pay $1M+ per answer IF it comes with a guaranteed solution or full cash back refund.

This is why being the top tier AI is so valuable.

janoc4• 1 year ago

Just going to put this here: https://www.wheresyoured.at/wheres-the-money/

cynicalpeace6• 1 year ago

"There Is No AI Revolution"

Good write-up.

But it focuses too much on the big companies. Many indiehackers have figured out how to make profit with AI:

1. No free tier. Just provide a good landing page.

2. Ship fast. Ship iteratively. Employ no one besides yourself.

3. Profit.

The old silicon valley idea that you need to raise a bunch of money, hire a bunch of devs, and scale a ton to satisfy investors is dying rapidly for software. You can code and profit millions as just a single person company, especially in the age of cursor.

sponnath• 1 year ago

If this were true wouldn't we be seeing a massive devaluation of software? (or alternatively increased demand for complex software)

janoc4• 1 year ago

And pretty much most of them just resell OpenAI/Anthropic/Google/Meta's APIs and model access, with something repackaged on top.

And none is remotely profitable.

tw1984• 1 year ago

this is the beginning of the end. OpenAI's lead is over.

strangescript• 1 year ago

This is just a bad model. I can't believe they released it. Yes it does have few interesting properties, but nothing that justifies the speed or cost when people are running R1 distillations on toasters for nothing.

ttul• 1 year ago

The high price is there to ensure nobody thinks of distilling their own cheap model using 4.5. OpenAI will undoubtedly distill a mini version themselves and they want to be out front for that benefit.

HackerThemAll• 1 year ago

And then I see people use it like this:

"Format the below JSON document for me"

<50KB of JSON pasted>

AI is already making people dumb, including IT folks.

iainctduncan• 1 year ago

So will we get people admitting they've been total jerks to Gary Marcus yet? Is he hyperbolic and over the top sometimes? Sure. Is he right about scaling not getting LLMs to AGI? Sure is looking like it.

I, for one, am so sick of listening to LLM fanboys wax on about "AGI" when they don't know the first goddamned thing about actual human cognition. For all his faults, Marcus studied human intelligence at a PhD level. I have only done a wee bit (music cognition as part of an interdisciplinary PhD I'm doing) and it's obvious to me, my supervisor (AI prof for 25 years) and anyone who knows anything about human cognition that LLMS are not going to get anywhere close to "thinking as well as a human" by scaling.

Bubble can't burst soon enough for me, sigh.

GPT-4.5

Related Stories