OpenAI’s board, paraphrased: ‘All we need is unimaginable sums of money’

https://daringfireball.net/2024/12/openai_unimaginable

by ajuhasz • 2 years ago

325 313 2 years ago

daringfireball.net

gnabgib• 2 years ago

Discussion (274 points, 2 days ago, 315 comments) https://news.ycombinator.com/item?id=42521744

greenthrow• 2 years ago

This is a link to an editorial by John Gruber, so not necessarily the same discussion.

amelius2• 2 years ago

So tired of people asking for giant piles of money that are high enough to corner a market.

minimaxir2• 2 years ago

And due to the lack of a moat, no amount of money can do that easily.

rchaud2• 2 years ago

...and claiming that it's for the good of society, and not personal self-enrichment.

ren_engineer• 2 years ago

and yet deepseek just created an amazing model with the fraction of their compute resources

stri8ed• 2 years ago

Trained at least in part on Chat-GPT data.

Handprint4469• 2 years ago

Doesn't matter. In fact, it makes it even funnier: all these investors spending billions of dollars on OpenAI just end up subsidizing the competing models.

rubslopes• 2 years ago

That was trained on data scrapped from the web. I'd say it's fair.

snats• 2 years ago

It's more of a distilled model, not a fair 1:1 comparison

janice1999• 2 years ago

Where is all the money going? Nvidia hardware?

rsrsrs86• 2 years ago

And compute.

jgalt2122• 2 years ago

Yes, but the most expensive CPU doesn't cost $40K.

smallmancontrov• 2 years ago

Spatulas in Jensen's decorative jar.

KennyBlanken• 2 years ago

That and power. The GB200 needs 1000W per GPU.

Eight racks of GB200's? You're talking about 1 megawatt of power. Well over that if you include PSU losses, cooling, and all the networking, storage, memory...

jazzyjackson• 2 years ago

IP Lawyers and Nuclear Lobbyists, presumably

tempodox• 2 years ago

Not to mention all the spin doctors they need to conjure up a regulatory moat in absence of a technical one.

esafak7• 2 years ago

If we get nuclear fusion and better GPUs as a byproduct of enabling LLMs, great. We haven't had this kind of dynamic since the heft of Windows pushed Intel to deliver faster processors.

ghshephard• 2 years ago

In all fairness regarding his comment about Netscape - anyone who invested in Netscape at the IPO (and certainly before the IPO) at $2.9B - made a ton of money on the Internet. On the last day of trading, after the AOL exit 4 years it was worth (cash, stock, etc...) around $10B. Thank Mike Homer for pushing that one through, and Jim Barksdale for being savvy enough to recognize it as the right play.

asadotzler8• 2 years ago

Also, they started from scratch, they didn't take NCSA's Mosaic public, they left the university, brought in some established players, created a new company, and re-wrote the entire browser from scratch. SamA wants to take everything the non-profit created and just make it for profit. If he want's to do that he can create a new company, lure away the talent, start from scratch, and run from there. This "it's just like Mosaic -> Netscape" line is total bullshit and either Gruber doesn't remember like I do or he's intentionally misleading for some other goal.

MndlshnDscpl• 2 years ago

Agreed. I think originally, it would have been more accurate to say IE was based on Mosaic. If I recall correctly, I think Microsoft bought out Spyglass Mosaic to base IE on, and that browser had been licensed from the NCSA. Netscape on the other hand had originally been Mosaic Communications(anyone remember home.mcom.com?) and changed the name when they did the clean rewrite. I think the name Mozilla came about because they were looking for a 'Mosaic Killer' or something along those lines. Memories are kind of fuzzy so I'm sure someone on here has a better recollection.

dheera• 2 years ago

Plot twist: OpenAI uses their GPU farms to mine cryptocurrency so that they get those unimaginable sums of money

jgalt2122• 2 years ago

That would be funny. Microstrategy with one level of abstraction.

asadotzler8• 2 years ago

Gruber's shilling some here. Andreesen et al. started over from scratch when they created Netscape. They didn't use any of the NCSA code. It was a ground up re-write.

SamA wants to take everything the non-profit built and use it directly in HIS for profit enterprise. Fuck him.

kragen2• 2 years ago

Who and what do you think Gruber is shilling for? I don't see Gruber saying anything that might tempt anyone to part from some of their money. He seems to be doing the diametrical opposite of shilling.

asadotzler8• 2 years ago

The "Silicon Valley can do no bad" crowd. He's making it sound like Altman's just following a well trod path here when they're clearly doing something novel and actually quite evil.

RajT882• 2 years ago

Well, who does Gruber usually shill for?

The question is - why would that one company want this hot take out there?

mvdtnz2• 2 years ago

What do you suppose he is shilling, and for whom?

I don't know who this Gruber guy is, but it's relieving to hear at least someone talk some sense about the ludicrous levels of investment being poured into something that most users go "heh that's neat" and would never consider paying for.

kridsdale12• 2 years ago

Gruber is OG. You should look him up.

asadotzler8• 2 years ago

He's in the "My guys can do now bad" gang. Always has been. Read his blog, it's 90% defending Big Tech and their abuses. He's especially bad about Apple, but all his angles are suspect.

scarface_745• 2 years ago

Outside of Apple circles, he’s best known for the inventor of Markdown.

raincole2• 2 years ago

He literally wrote this:

OpenAI’s board now stating “We once again need to raise more capital than we’d imagined” less than three months after raising another $6.6 billion at a valuation of $157 billion sounds alarmingly like a Ponzi scheme

Ponzi scheme. If he's shilling something I guess it's "shorting".

epolanski• 2 years ago

I think their vision is as it should be ambitious, but I don't believe they can gain any real technical moat.

Models are commodities, even in the case Open ai goes through another major breakthrough nothing can stop some of their employees to run to other companies or founding their own and replicating or bettering the OpenAI results.

In fairness I realize that I don't use any of OpenAI's models. There are better alternatives for coding, translating or alternatives that are simply faster or cheaper (Gemini) or more open.

brookst5• 2 years ago

Technical moats are the worst ones. The absolute best you can hope for is patents, but those take years to issue and then you have to fight after tbe fact. Trade secrets are nice but hard to defend.

The best moats are scale (Walmart), brand (“Google” means search), network effects (Facebook, TikTok). None of those are perfect but all are better than just having better tech.

otherme123• 2 years ago

Facebook and TikTok could collapse in under a year, that has happened before. Network effects seems very brittle: AIM or Fotolog doesn't exist, and they were huge. MySpace is a shadow of what it was.

OTOH, WD40 has a technical moat since 1953, without a patent. There are a number of companies who rely on technical moat mixed with excellent technical image: Makita or Milwaukee Tool come to mind. You can have also a company based on technology moat that theoretically shouldn't exist (patents expired, commodity products), but it does (Loctite, 3M).

numpy-thagoras3• 2 years ago

OpenAI's engineers: brilliant, A+, A* even.

OpenAI's C-suite: Well, they earned the C, but it was a letter grade.

What a profoundly unimaginative strategy. No matter the industry, a large-scale diversion of resources towards a moonshot goal will likely get you to that goal. They haven't made an argument as to why we should do that just for them, especially with all of the other alternatives.

And no, advertising your previously secret testing models (e.g. o3) as if they were market competitors is not how to prove they should have our money.

abc-1• 2 years ago

OpenAI's engineers: brilliant, A+, A* even.

Then why do they look about average to Anthropic, Mistral, DeepSeek, and many others of that cohort despite having x100 the amount of resources?

numpy-thagoras3• 2 years ago

Praising one doesn't mean condemning another.

ryanwaggoner• 2 years ago

”They haven’t made an argument as to why we should do that just for them…”

What “we”? They’ve already raised billions, and I suspect they’re about to succeed at raising tens of billions more, despite the skepticism of random HN users.

numpy-thagoras3• 2 years ago

Tens of billions? Sama has been talking about how their ask can be for over a trillion dollars invested. What "we"? That trillion is going to come out of our pockets one way or another, either as costs passed down, as government handouts from our tax dollars, as redirected investments that dry up opportunities for SWEs elsewhere, etc.

robertlagrant4• 2 years ago

This doesn't seem to say much to back its comparisons. Wouldn't spending a giant amount of money create some sort of moat?

wmf5• 2 years ago

Not if your competitors are spending the same amount (or spending vastly less to get similar results in the case of DeepSeek).

mvdtnz2• 2 years ago

From what I'm reading of the industry, no not really. The laughably huge investments seem to buy OpenAI approximately a 6 month head start over open source models, which is not much.

AnimalMuppet• 2 years ago

Depends. If they manage to actually find a way to create an AGI, then a six month head start might be massive at that point. Until then... not so much.

robertlagrant4• 2 years ago

Why is this, though? Are they wasting the money? Or are their discoveries instantly disseminated to their competitors? Or are the open source models just currently sponsored by rich people?

patrickhogan15• 2 years ago

I agree that at its current rate of $20 per month and continuous cash burn, OpenAI’s model doesn’t seem sustainable. But the same could have been said about Google in its early days. Back then, Google spent heavily to index websites without a clear path to profitability—until it figured out how to monetize. Similarly, OpenAI will likely release more products to generate revenue.

OpenAI has the potential to create the next groundbreaking innovation, like the iPhone. Who needs apps when the AI itself can handle everything?

VHRanger• 2 years ago

Google spent heavily to index websites without a clear path to profitability

Uh, they were advertising in 1999 already?

patrickhogan15• 2 years ago

Please read the academic paper by Larry Page and Sergey Brin, available at this link: https://blogs.cornell.edu/info2040/2019/10/28/the-academic-p...

You might find their perspective on advertising surprising.

Returning to my earlier point, please expand your argument about why OpenAI may fail. Let’s aim to elevate the discussion.

patrickhogan15• 2 years ago

If you’re going to downvote, please consider leaving a comment—I’d really value the feedback.

I’d like to see more thoughtful discussion here. The debate often feels stuck between oversimplified takes like “It’s just a commodity” and “No, it’s a game changer!” Let’s assume for a moment that OpenAI operates as a commodity business but still figures out how to become profitable, even with small margins. After all, US Steel was a commodity and became hugely successful.

Given these assumptions, why do you think OpenAI might not succeed? Let’s move the conversation forward.

lesuorac3• 2 years ago

Back then, Google spent heavily to index websites without a clear path to profitability

[citation needed]

Google's goal was always advertising just like every other search engine.

wmf5• 2 years ago

Google didn't have ads until 2000. They strongly resisted banner ads (which were the only format at the time) until they invented contextual text ads.

patrickhogan15• 2 years ago

Please read the academic paper by Larry Page and Sergey Brin, available at this link:

https://blogs.cornell.edu/info2040/2019/10/28/the-academic-p...

You might find their perspective on advertising surprising.

Returning to my earlier point, please expand your argument about why OpenAI may fail. Let’s aim to elevate the discussion.

cs7022• 2 years ago

This rings true to my ears:

There is no technical moat in this field, and so OpenAI is the epicenter of an investment bubble. Thus, effectively, OpenAI is to this decade’s generative-AI revolution what Netscape was to the 1990s’ internet revolution. The revolution is real, but it’s ultimately going to be a commodity technology layer, not the foundation of a defensible proprietary moat. In 1995 investors mistakenly thought investing in Netscape was a way to bet on the future of the open internet and the World Wide Web in particular.

OpenAI has a short-ish window of opportunity to figure out how to build a moat.

"Trying to spend more" is not a moat, because the largest US and Chinese tech companies can always outspend OpenAI.

The clock is ticking.

neals• 2 years ago

I get the moat idea. But are there really any moats? What's a good moat these days anyway? Isn't being first and "really good" fine as well?

kridsdale12• 2 years ago

Network effects are a moat.

A vertically integrated system that people depend on with non-portable integrations is a moat.

Regulatory Capture is a moat.

ninth_ant• 2 years ago

The Network effect, and cultural mind share are two pretty effective moats.

Meta and X proven surprisingly resilient in the face of pretty overwhelming negative sentiment. Google maintains monopoly status on Web Search and in Browsers despite not being remarkably better than competition. Microsoft remains overwhelmingly dominant in the OS market despite having a deeply flawed product. Amazon sells well despite a proliferation of fake reviews and products. Netflix thrives even while cutting back sharply on product quality. Valve has a near-stranglehold on PC games distribution despite their tech stack being trivially replicable. The list goes on.

GolfPopper• 2 years ago

Here's an idea for a moat: "You know all the ethically extremely questionable grabbing of personal data and creative work we did to build our LLM? Yeah, that's illegal now. For anyone else to do, that is."

otterley3• 2 years ago

toddmorey• 2 years ago

Genuinely curious if anyone has ideas how an LLM provider could moat AI. They feel interchangeable like bandwidth providers and seem like it will be a never ending game of leapfrog. I thought perhaps we’d end up with a small set of few top players just based on the scale of investment but now I’ve also seen impressive gains from much smaller companies & models, too.

wmf5• 2 years ago

You'd have to have either a breakthrough algorithm (that you keep secret forever) or proprietary training data.

scarmig3• 2 years ago

1) Push for regulations around data provenance. If you train on anything, you have to prove that you had the rights to train on it. This would kill all but the largest providers in the USA, though China would still pose a problem. You could work around that bit by making businesses and consumers liable for usage of models that don't have proof of provenance.

2) If you had some secret algorithm that substantially outperformed everyone, you could win if you prevented leakage. This runs into the issue that two people can keep a secret, but three cannot. Eventually it'll leak.

3) Keep costs exceptionally low, sell at cost (or for free), and flood the market with that, which you use to enhance other revenue streams and make it unprofitable for other companies to compete and unappealing as a target for investors. To do this, you have to be a large company with existing revenue streams, highly efficient infrastructure, piles of money to burn while competitors burn through their smaller piles of money, and the ability to get something of value from giving something out for free.

lolinder• 2 years ago

Regulatory capture. Persuade governments that AI is so dangerous that it must be regulated, then help write those regulations to ensure no one else can follow you to the ladder.

That's half the point of OpenAI's game of pretending each new thing they make is too dangerous to release. It's half directed at investors to build hype, half at government officials to build fear.

marcyb5st• 2 years ago

An algorithmic breakthrough IMHO. If someone finds out either how to get 10x performance per parameter or how to have a model actually model real causality (to some degree) they will have a moat.

Also, I suspect that the next breakthrough will be kept under wraps and no papers will be published explaining it.

Jyaif• 2 years ago

curious if anyone has ideas how an LLM provider could moat AI

By knowing a lot about me, like the details of my relationships, my interests, my work. The LLM would then be able to be better function than the other LLMs. OpenAI already made steps in that direction by learning facts about you.

By offering services only possible by integrating with other industries, like restaurants, banks, etc... This take years to do, and other companies will take years to catch up, especially if you setup exclusivity clauses. There's lots of ways to slow down your competitors when you are the first to do something.

mmaunder• 2 years ago

DeepSeek’s recent progress with V3 being a case in point which reportedly only cost $5.5M.

phodo2• 2 years ago

One view is that it’s not first mover, but first arriver advantage. Whoever gets to AgI (the fabled city of gold or silver?, Ag pun intended) will achieve exponential gains from that point forward, and that serves as the moat in the limit. So you can think of it as buying a delayed moat, with an option price equivalent to the investment required until you get to that point in time. Either you believe in that view or you don’t. It’s more of an emotional / philosophical investment thesis, with a low probability of occurrence, but with a massive expected value.Meanwhile, consumers and the world benefit.

malux85• 2 years ago

Legislation, if you can’t compete on merit then regulate.

CamperBob2• 2 years ago

Genuinely curious if anyone has ideas how an LLM provider could moat AI.

Patents. OpenAI already has a head start in the game of filing patents with obvious (to everybody except USPTO examiners), hard-to-avoid claims. E.g.: https://patents.google.com/patent/US12008341B2

btilly• 2 years ago

Three ideas. All tried, none worked yet.

First, get government regulation on your side. OpenAI has already looked for this, including Sam Altman testifying to Congress about the dangers of AI, but didn't get the regulations that they wanted.

Second, put the cost of competing out of reach. Build a large enough and good enough model that nobody else can afford to build a competitor. Unfortunately a few big competitors keep on spending similar sums. And much cheaper sums are good enough for many purposes.

Third, get a new idea that isn't public. For instance one on how to better handle complex goal directed behavior. OpenAI has been trying, but have failed to come up with the right bright idea.

minimaxir2• 2 years ago

One way to build a technical moat is to build services which encourage lock-in, and therefore make it hard to switch to a competitor. Some of OpenAI's product releases help facilitate that: the Assistant API creates persistent threads that cannot be exported, and their File Search APIs build vector stores that cannot be exported.

brookst5• 2 years ago

Create value at a higher layer and depend on tight integration to generate revenue and stickiness at both layers. See: Windows + MS Office.

They don’t need to make a moat for AI, they need to make a moat for the OpenAI business, which they have a lot of flexibility to refactor and shape.

seanmcdirmid6• 2 years ago

Create closed source models that are much better than the other ones, don't publish your tricks to obtaining your results, and then lock them down in a box that can't be tampered with? I hope it doesn't go that way.

Alternatively, a model that takes a year and the output of a nuclear power plant to train (and then you can tell them about your tricks, since they aren't very reproducible).

shalmanese• 2 years ago

In the early days of Google, people believed there could be absolutely no moat in search because competition was just "one click away" and even Google believed this and deeply internalized this into their culture of technological dominance as the only path to survival.

At the beginning of ride sharing, people believed there was absolutely no geographical moat and all riders were just one cheaper ride from switching so better capitalized incumbents could just win a new area by showering the city with discounts. It took Uber billions of dollars to figure out the moats were actually nigh insurmountable as a challenger brand in many countries.

Honestly, with AI, I just instinctively reach for ChatGPT and haven't even bothered trying with any of the others because the results I get from OAI are "good enough". If enough other people are like me, OAI gets order of magnitudes more query volume than the other general purpose LLMs and they can use that data to tweak their algorithms better than anyone else.

Also, current LLMs, the long term user experience is pretty similar to the first time user experience but that seems set to change in the next few generations. I want my LLM over time to understand the style I prefer to be communicated in, learn what media I'm consuming so it knows which references I understand vs those I don't, etc. Getting a brand new LLM familiar enough to me to feel like a long established LLM might be an arduous enough task that people rarely switch.

adamnemecek• 2 years ago

Spend money developing proprietary approaches to ML.

bruce5112• 2 years ago

There is no technical moat, but that doesn't mean there isn't a moat.

Amazon might be a good analogy here. I'm old enough to remember when Amazon absorbed billions of VC money, making losses year over year. Every day there was some new article about how insane it was.

There's no technical moat around online sales. And lots of companies sell online. But Amazon is still the biggest (by a long long way) (at least in the US). Their "moat" is in public minds here.

Google is a similar story. As is Facebook. Yes the details change, but rhe basic path is well trodden. Uber? Well the juries still out there.

Will OpenAI be the next Amazon? Will it be the next IBM? We don't know, but people are pouring billions in to find out.

chubot2• 2 years ago

Amazon does have a technical moat - fast shipping and warehouse operations. It’s not something that Walmart or Target can replicate easily.

Retail margins are razor thin, so the pennies of efficiency add up to a moat

tehjoker• 2 years ago

Amazon's moat is their logistics network, comprehensive catalogue, prices, and value adds (like Prime Video).

dewitt• 2 years ago

“Amazon might be a good analogy here. I'm old enough to remember when Amazon absorbed billions of VC money, making losses year over year. Every day there was some new article about how insane it was.”

Not to take away from the rest of your points, but I thought Amazon only raised $8m in 1995 before their IPO in 1997. Very little venture capital by today’s standard.

gloryjulio• 2 years ago

There's no technical moat around online sales. And lots of companies sell online. But Amazon is still the biggest (by a long long way) (at least in the US). Their "moat" is in public minds here.

I don't think that's true. I think it's actually the opposite. Global physical logistic is way harder than software to scale. That's Amazon's moat

ripped_britches• 2 years ago

This is an odd thing to say.

Facebook has an enormous network effect. Google is the lynchpin of brokering digital ads to the point of being a monopoly. Someone else mentioned Amazons massive distribution network.

morgante• 2 years ago

Amazon has scale economics, branding power, and process power. It would take years to fully rebuild Amazon from scratch even given unlimited money.

Right now, OpenAI's brand is actually probably its strongest "moat" and that is probably only there because Google fumbled Bard so badly.

lotsofpulp5• 2 years ago

Amazon might be a good analogy here. I'm old enough to remember when Amazon absorbed billions of VC money, making losses year over year. Every day there was some new article about how insane it was.

When was this true? Amazon was founded Jul 1994, and a publicly listed company by May 1997. I highly doubt Amazon absorbed billions of dollars of VC money in less than 3 years of the mid 1990s.

https://dazeinfo.com/2019/11/06/amazon-net-income-by-year-gr...

As far as I can tell, they were very break even until Amazon Web Services started raking it in.

raincole2• 2 years ago

Facebook and ChatGPT are the opposite kinds of products.

saurik• 2 years ago

A couple other comments touch on the point I want to make, but I feel they don't nail it hard enough: if today you told me you had a website that was 100% technologically identical to Amazon, but you were willing to make slightly less money, you'd still have no products... as a user I'd have no reason to go there, but then without users there is no reason to sell there, so you have a circular bootstrapping issue. That is a moat.

This is very different from OpenAI: if you show me a product that works just as well as ChatGPT but costs less--or which costs the same but works a bit better--I would use it immediately. Hell: people on this website routinely talk about using multiple such services and debate which one is better for various purposes. They kind of want to try to make a moat out of their tools feature, but that is off the path of how most users use the product, and so isn't a useful defense yet.

noch5• 2 years ago

There is no technical moat, but that doesn't mean there isn't a moat.

Gruber writes:

" My take on OpenAI is that both of the following are true:

OpenAI currently offers, by far, the best product experience of any AI chatbot assistant. There is no technical moat in this field, and so OpenAI is the epicenter of an investment bubble. "

It's amusing to me that he seems to think that OpenAI (or xAI or DeepSeek or DeepMind) is in the business of building "chatbots".

The prize is the ability to manufacture intelligence.

How much risk investors are willing to undertake for this prize is evident from their investments, after all, these investors all lived through prior business cycles and bubbles and have the institutional knowledge to know what they're getting into, financially.

How much would you invest for a given probability that the company you invest in will be able to manufacture intelligence at scale in 10 years?

chii4• 2 years ago

Their "moat" is in public minds here.

no, the amazon moat is scale, and efficiency, which leads to network effect. The chinese competitors are reaching similar scales, so the moat isn't insurmountable - just not for the average mom and dad business.

scarface_745• 2 years ago

Amazon spent those billions on warehouses, server, its own delivery network, etc.

Anyone can sell online. But not just anyone has those advantages like same day abs next day shipping

csomar• 2 years ago

There is no technical moat, but that doesn't mean there isn't a moat.

The moat is actually huge (billions of $$). What is happening is that there are people/corps/governments that are willing to burn this kind of money on compute and then give you the open weight model free of charge (or maybe with very permissive and lax licensing terms).

If it wasn't for that, there will be roughly three players in the market (Anthropic and recently Google)

huijzer• 2 years ago

Their "moat" is in public minds here.

Same for OpenAI. Anytime I talk to young people who are not programmers, they know about ChatGPT and not much else. Never heard of Llama nor what an LLM is.

benterix2• 2 years ago

It is not a good analogy because services like Facebook, Amazon, Instagram depend on a critical mass of users (sellers, content sharers, creators, etc.). With SaaS this is not a crucial component as the service is generated by software, not other users.

conartist6• 2 years ago

Amazon built a bit of a moat too: for example I happen to know that they own a patent on the photography setup that allows them to capture consistent product photos at scale.

DoctorOetker• 2 years ago

Where is the actual full statement and call for funds by the board?

mring336212• 2 years ago

Amazon's moat-ish stickiness:

1) Almost the best price, almost all the time

2) Reliably fast delivery

3) Reliably easy returns

4) Prime memberships

jsjohnst• 2 years ago

Amazon might be a good analogy here. I'm old enough to remember when Amazon absorbed billions of VC money, making losses year over year.

Apparently old enough to forget the details. I highly recommend refreshing your memory on the topic so you don’t sound so foolish.

1. Amazon had a very minimal amount of VC funding (less than $100M, pretty sure less than $10M)

2. They IPO’d in 1997 and that still brought in less than $100M to them.

3. They intentionally kept the company at near profitability, instead deciding to invest it into growth for the first 4-5yrs as a public company. It’s not that they were literally burning money like Uber.

4. As further proof they could’ve been profitable sooner if they wanted, they intentionally showed a profit in ~2001 following the dotcom crash.

Edit: seems the only VC investment pre-IPO was KP’s $8M. Combine that with the seed round they raised from individual investors and that comes in under $10M like I remembered.

mrtksn• 2 years ago

In 1995 investors mistakenly thought investing in Netscape was a way to bet on the future

Maybe they were just too early, later on it turned out that the browser is indeed a very valuable and financially sound investment. For Google at least.

So having a dominant market share can indeed be even if the underlying tech is not exactly unobtainable by others.

rchaud2• 2 years ago

A browser would be worthless to Google without DoubleClick ad network monopoly, which they acquired years before Chrome. Netscape tried to charge for their browser and stopped because Microsoft was giving away Internet Explorer free with every Windows PC.

chii4• 2 years ago

OpenAI has a short-ish window of opportunity to figure out how to build a moat.

and they are probably going to go with regulatory moat over technical moat.

Nasrudith• 2 years ago

I guess that explains all of the 'our product is an existential threat' chuunibyou bullshit. Trying to get a regulatory moat instead.

florakel• 2 years ago

Why is everyone so sure that technology can’t be the moat? Why are you convinced that all AI systems will be very similar in performance and cost, hence be interchangeable? Isn’t Google the perfect example that you can create a moat by technological leadership in a nascent space? When Google came along it was 10X better than any alternative and they grabbed the whole market. Now their brand and market position make it almost impossible to compete. I guess the bet is that one of the AI companies achieves a huge breakthrough and a 10X value creation compared to its competitors…

amelius2• 2 years ago

Maybe because for 99.9% of users (usecases) what today's llm technology offers is already good enough?

Or maybe nvidia has the moat. Or silicon fabs have it.

esafak7• 2 years ago

ChatGPT isn't ten times better than Gemini or Claude, is it? Even if they magically released such a model, the competition would quickly catch up. The competition has similar or better resources.

BeefWellington• 2 years ago

OpenAI has a short-ish window of opportunity to figure out how to build a moat.

See the scramble to make themselves arbiters of "good AI" with regulators around the world. It's their only real hope but I think the cat's already out of the bag.

fmap• 2 years ago

Adding to this, inference is getting cheaper and more efficient all the time. The investment bubble is probably the biggest reason why inference hardware is so expensive at the moment and why startups in this sector are only targeting large scale applications.

Once this bubble bursts, local inference will become even more affordable than it already is. There is no way that there will be a moat around running models as a service.

---

Similarly, there probably won't be a "data moat". The whole point of large foundation models is that they are great priors. You need relatively few examples to fine tune an LLM or diffusion model to get it to do what you want. So long as someone releases up to date foundation models there is no moat here either.

n_ary2• 2 years ago

I believe that, their brand “ChatGPT” is the moat. Here in EU, every news media and random strangers on the streets who don’t understand anything beyond TikTok also knows that AI=ChatGPT. The term is so ubiquitous with AI that, I am willing to bet when someone tells someone else about AI, they tell ChatGPT and the new person will search and use ChatGPT without any knowledge of Claude/Gemini/LLama et. al. This is the same phenomenon as web search is now = Google, despite other prominent ones existing.

The competitions are mostly too narrow(programming/workflow/translation etc) and not interesting.

otterley3• 2 years ago

A brand isn't much of a moat: MySpace's brand was a moat for a few years. Then Facebook came and ate their lunch.

boh• 2 years ago

Honestly the more I use ChatGPT, the more I see of their "moat-ish" opportunities. While the LLM itself may be ultimately reproducible, the user experience needs a UI/UX wrapper for most users. The future is not user exposure to "raw" LLM interactions but rather a complex workflow whereby a question goes through several iterations of agents, deterministic computation using "traditional" processing, and figuring out when to use what and have it all be seamless. ChatGPT is already doing this but it's still far from where it needs to be. It's totally possible for a company to dominate the market if they're able to orchestrate a streamlined user experience. Whether that will ultimately be OpenAI is an open question.

cyanydeez2• 2 years ago

....or they could bewikipedia.

But greeed isgood!

yalogin• 2 years ago

This is what I have been asking people in the know too. It seems like developing this is straight forward. So not sure why OpenAI has a lead. That too with how fast things are, the gap should be instantly closed.

JojoFatsani• 2 years ago

Same

bloomingkales2• 2 years ago

Unimaginable sums of money can only come from oil rich countries. They are probably just mentally trying to accept that they are going to take the money from these nations.

The word "open" is still under threat in this scenario too.

rsync• 2 years ago

You mean like the United States?

The world‘s largest oil producer?

bloomingkales2• 2 years ago

Lol, I meant the muslims.

wmf5• 2 years ago

I see some people talking about OpenAI "stealing" IP from the nonprofit side, but remember that they went "capped profit" back in 2019. It's possible that GPT-3 and later were developed entirely by the capped-profit side anyway. I doubt there's much if any pre-2019 IP left.

onlyrealcuzzo• 2 years ago

There is no profit yet...

Jensson2• 2 years ago

But they are expected to make profits in the future, that is what investments into for-profits are for.

asadotzler8• 2 years ago

Did they license and pay for all of the non-profit work they utilized in GPT-3? OR are you claiming it was a clean-room re-write? I think you're pretty naive if you think the latter happened.

wmf5• 2 years ago

I think they could have done just enough of both to plausibly neutralize Elon's lawsuit.

bravura• 2 years ago

People keep saying stuff like this, but I don't believe it: "There is no technical moat in this field, and so OpenAI is the epicenter of an investment bubble."

AI progress is driven by strong valuable data. Detailed important conversation with a chatbot are much more valuable that quick search queries. As LLMs extend past web UIs, there is even more interaction data to capture and learn from.

The company that captures the most human-AI interaction data will have a TREMENDOUS moat.

ijustlovemath4• 2 years ago

What do you think they've been building these multimodal models with? If I had Google application logs from the past few decades I would absolutely be transforming it and loading into my training dataset. I would not be surprised if Google/Meta/Msft are not doing this already

When the big companies say they're running out of data, I think they mean it literally. They have hoovered up everything external and internal and are now facing the overwhelming mediocrity that synthetic data provides.

skydhash4• 2 years ago

Digital data are only a tiny part of the influx of information that people interact with. It's the same error platforms made with releasing movies straight to their services. Yes, people watch more movies at home, but going to cinemas is a whole experience that encompasses more than just watching a movie. Yes, books are great, but traveling and tutoring is much more impactful.

mrbungie2• 2 years ago

Data is becoming a commodity in this regard. That can't be really their moat when Google, Anthropic, etc are publishing similar products.

asadotzler8• 2 years ago

Well, once the powerful steal it all, of course it becomes commoditization. Had they been required to abide by the law, something that few Silicon Valley VC-backed companies really worry about, this kind of "information" would not be a commodity, it'd be a lucrative property. But, Silicon Valley doesn't give a fuck about anyone so they just stole it all and now it's basically worthless as a result.

paxys• 2 years ago

Yup. We have seen time and again that companies don't need a "technical moat" to stay in the lead. First mover advantage and a 1-2 year head start is always enough. Of course they also need to keep their foot on the gas and not let their product get overtaken. With all the talent OpenAI has I'm pretty sure they will manage.

motoxpro• 2 years ago

When will this come to pass? OpenAI has many orders of magnitude more conversational data and Anthropic just keeps catching up. Until there is some evidence (OpenAI winning, or Google winning rather than Open Source catching up), I don't belive this is true.

staticman2• 2 years ago

Is conversation data really that valuable?

If I have the LLM translate a text from French to English... what is there to learn from that? Maybe the translation is great maybe it's awful, but there's no "correct" translation to evaluate the LLM against to improve it.

If I ask the chatbot for working code and it can't provide it, again, there's no "correct" code to train it against found in the conversation.

If I ask an LLM to interpret a bible passage, whether it does a good job or a terrible job there's no "correct" answer that the provider has to use as the gold standard, just the noise of people chatting with arbitrary answers.

resonious• 2 years ago

The paraphrasing is very comical, but a bit different from what they actually said, which is more like "more than we had imagined".

thefz• 2 years ago

Well it's Gruber, not really known for being a decent blogger. He's the most biased person you can get acting like a prophet on the internet.

YetAnotherNick• 2 years ago

There is no technical moat in this field

This is getting so repetitive now that it is stated as a truism.

Isn't it the same bet yahoo was betting on in 2000 that it would win because their product branding is better? And now, Yahoo's and Microsoft's search engine is worse than Google from 2 decades ago.

scarmig3• 2 years ago

Google's search engine is worse than Google from 2 decades ago.

otterley3• 2 years ago

Is it Google's search engine, or is it that there was less pollution on the Internet two decades ago?

jsheard2• 2 years ago

Correction: all they need to succeed is unimaginable sums of money and the ironclad right to assimilate every work ever created by humanity without compensation. Who could refuse such a modest request?

https://www.theguardian.com/technology/2024/jan/08/ai-tools-...

brookst5• 2 years ago

So why do they pay to license so much content? And you can’t really be suggesting that public domain should exclude AI training?

jsheard2• 2 years ago

That's a good question, why do they choose to pay for certain content while also continuing to insist that all publicly available content should be fair game? It's an incoherent stance, and the only plausible explanation I can come up with is that those "licensing deals" are out-of-court legal settlements in a trenchcoat.

xboxnolifes2• 2 years ago

To sell the illusion of not using copyrighted works they make it publicly aware that they license some works.

ryao2• 2 years ago

I once had a colleague that told me in order to make a small fortune, all you need to do is begin with a large fortune.

sumedh3• 2 years ago

Was your colleague Richard Branson?

ryao2• 2 years ago

No.

jmclnx• 2 years ago

AI going down the Cold Fusion path ? Not much of a surprise to me

myhf• 2 years ago

Unimaginable Sums of Money Are All You Need

The Unreasonable Effectiveness of Huge Sums of Money in the Natural Sciences

Unimaginable Sums of Money Considered Harmful

seydor• 2 years ago

All we need is unreasonably effective paper titles

theideaofcoffee• 2 years ago

How I Learned to Stop Worrying and Love Unimaginable Sums of Money

yobbo• 2 years ago

Unimaginable Sums of Money: For Fun And Profit

SignalsDoctor• 2 years ago

The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Unimaginable Sums of Money

eptcyka• 2 years ago

Worst person one knows has a legitimately good opinion.

andrewstuart• 2 years ago

> OpenAI currently offers, by far, the best product experience of any AI chatbot assistant.

Claude is noticeably better

hebrox• 2 years ago

And using Claude with Cline, partially because of prompt caching, it's noticeably cheaper as well.

dmazin2• 2 years ago

I use Claude because the model is indeed better, but having listened to John's podcast I know that he means that the experience of using OpenAI overall is better.

tiffanyh• 2 years ago

defensible moat … investors mistakenly thought investing in Netscape was a good way to bet on the future

Yet Chrome for Google did help create a moat.

A moat that’s is so strong the DoJ is investigating if Chrome should be a forced divesture from Google/Aplhabet.

Note: I do generally agree with the article, but this also shows why you shouldn’t use analogies to reason.

scarface_745• 2 years ago

Google was already dominant before Chrome. Why would anyone want to buy Chrome instead of just basing a browser on Chromium? How are they going to monetize it?

LittleTimothy• 2 years ago

I don't think you're analyzing this right. The web browser itself has no defensible moat. The reason that Chrome is winning is because Google has a massive money printer in the basement run by 3 guys and a hungry Alsatian (trained to bite anyone who messes with the money printer), and then 100,000 smart engineers desperately trying to find some application to throw that money at.

The result is Chrome, Google invests in it because they don't want to be intermediation between them and eyeballs - it's a strategic play, not necessarily a direct money maker. So that's why they do it. But there is no moat, they have the number 1 browser because they spend enormous sums of money hiring top engineers to build the number 1 browser.

And this has been true of the browser for a long time, Internet Explorer won because they forced people to use it and used a tonne of anti-competitive practices to screw the competition. Firefox still managed to disrupt that simply by building a better product. This is the clear evidence there is no moat - the castle has been stormed repeatedly.

alexey-salmin• 2 years ago

The moat for Chrome is "it's free because it's subsidized by Google's search monopoly". Before that moat for Explorer was "it's free becaus it's subsidized by Microsoft's OS monopoly", that's what killed Netscape, they couldn't subsidize the Navigator.

OpenAI can't do anything comparable to Google at this point because they have no other product. If anything they're more like Netscape.

benterix2• 2 years ago

Frankly, OpenAI efforts seem quite funny for me personally as for most tasks I do Claude is far superior and yet OpenAI behaves as if they were the only game in town.

The so-called open source models are getting better and better and even if OpenAI suddenly discovered some new tech that would allow for another breakthrough, it will be immediately picked up by others.

sumedh3• 2 years ago

yet OpenAI behaves as if they were the only game in town.

The data backs it up, Anthropic make most of their revenue from API while ChatGpt makes most of its revenue from the plus plan.

TaurenHunter• 2 years ago

'unimaginable sums of money’ considered harmful.

Gys• 2 years ago

A perfect fit for Softbank. I do not understand why a 'dream deal' between them has not happened yet?

yodsanklai2• 2 years ago

How do they define success after we gave them all the capital they're requesting?

fragmede• 2 years ago

According to 'The Information', AGI will be achieved only when OpenAI's AI systems generate profits exceeding $100 billion, so I'm guessing, for them, AGI = success.

niemandhier• 2 years ago

Investors observed the pattern that the largest platform eats the market and bet on openAI moving along the same paths.

I disagree, that phenomenon is tied to the social network like phenomena we saw with WhatsApp and Facebook and to the aggregator business model of Amazon and Google.

Mathematically we can describe the process by which these monopolies form as nodes joining graphs:

Each node selects graphs to connect to in such a way that the probability is proportional to the number of nodes in the graph.

Sure Amazon and google feature two types of nodes, but the observation still fits: Selling on Amazon makes sense if there are many customers, buying on Amazon makes sense if there are many offers.

OpenAIs business does not have this feature, it does not get intrinsically get more attractive by having more users.

netdevphoenix2• 2 years ago

This is why I don't think LLMs will lead anywhere other than the present: they are very expensive. If top LLM company in the world can't bring those costs down in a sensible amount of time, it tells you everything you need to know about the future of LLMs. Rest assured, once OpenAI goes down and it really sounds like they are way past the peak, valuations for other LLMs companies will follow. And once they can't raise funding, all of them aside from those backed by big conglomerates (see Google's Gemini and Facebook's Llama) will survive.

The long term future of LLMs sure looks like the LLMs themselves will be commodities and the real value will lie in the use of those LLMs to deliver value

ahel• 2 years ago

Why Meta decided to open source their Llama architecture and complete models? From a strategic business perspective, I'm trying to puzzle why they would give that up, a competitive advantage (even though it might not be as good as OpenAI product), rather than directly competing with OpenAI. What's the reasoning behind this decision?

eden-u4• 2 years ago

meta is large enough (and diversified enough), when OpenAI valuation goes to pennies (because LLM won't solve any real problem, nor they'll remain relevant in the long term and open source alternative are more than enough for most chatbot usecases), they can acquire all the relevant technologies from OpenAI, Anthropic and such. The real question is why google didn't release their models.

yodsanklai2• 2 years ago

because LLM won't solve any real problem,

LLMs in their current form are very useful tools for many people. Most programmers I know use them daily. I have even friends who aren't tech savvy who paid the subscription to ChatGPT and use it all the time.

apbytes• 2 years ago

I'd imagine that Meta is trying to avoid an AI / LLM monopoly from happening as the primary goal. They suffered when Apple was a gateway to their service for ios users and decided to shut their access to user data off. And AI is clearly going to be used widely to aid in content generation if not doing entirely. Also they tried jumping on the chatbot hype with M or something. That didn't pan out as well. By opening up AI, they would enable tools that allow users to pump out more content easily and spend more time in app. Getting the goodwill of dev community is a bonus. A brilliant move from Meta.

dmazin2• 2 years ago

Meta's is doing classical (and brilliant) "commodify your complements".

LLMs and AI is not where Meta adds value to the market. They add value via advertising. Therefore, AI is not Meta's product, and thus Meta should aim to make AI as cheap as possible.

They did the same thing for infra. Meta doesn't sell cloud computing, so they want to make infra as cheap as possible.

joseneca• 2 years ago

Don't we all...

betimsl• 2 years ago

Don't we all?

OpenAI’s board, paraphrased: ‘All we need is unimaginable sums of money’

Related Stories