Amidst the noise and haste, Google has successfully pulled a SpaceX

https://markmaunder.com/2025/amidst-the-noise-and-haste-google-has-successfully-pulled-a-spacex/

by mmaunder • 2 years ago

69 60 2 years ago

lmm3• 2 years ago

OK but vertical integration and low costs is only half of the problem. You also have to make a product that people want, otherwise the whole enterprise is meaningless. SpaceX is selling launches every week. What is Google selling that uses all those TPUs, and to whom?

lokar• 2 years ago

Behavioral advertising.

dekhn2• 2 years ago

TPUs are used internally for products and research and also sold as a service thru Google Cloud. The list of TPU customers is pretty long. But Google could the product entirely internal and still reap plenty of benefit.

mmaunder6• 2 years ago

Gemini 2 was entirely trained on Trillium, Googles 6th generation TPU. About 100,000 of them. And all inference is run on TPUs.

https://venturebeat.com/ai/google-new-trillium-ai-chip-deliv...

lmm3• 2 years ago

Right. But are people paying for Gemini? Does all that vertical integration make it cheaper than its competitors? (Which is what works for SpaceX)

Reubend• 2 years ago

Even if the LLM hype dies, and nobody at all wants to rent TPUs from GCP, then Google can use the TPUs to train and run large advertising models and improve the quality of their search.

They themselves are a big enough customer to make this worth it. Any business from GCP is just icing on the cake. Keep in mind that TPUs can train non-LLM models too.

m_mueller3• 2 years ago

All the ad tech in the world won’t matter if people have stopped using the medium those ads are displayed on top of by then. I think the way people use the web is about to change drastically.

jeffbee• 2 years ago

Google has been charging people actual money to access features running on TPUs for years. I helped launch TPU inference features for Gmail in 2015. There may have been earlier systems running on TPU that I wasn't aware of.

griomnib2• 2 years ago

Targeted ads use a lot of behavioral models. Google sells ads.

lmm3• 2 years ago

Has this improved their ad targeting? In a way that shows up in the bottom line, or even the top line?

kelvinjps10• 2 years ago

Just killing the competition is enough with their integrated products?, like google search with ai features,(perplexity.ai) Ai on android now gemini replacing google voice (chat gpt and others).

creato• 2 years ago

Many of those launches are sold to itself (starlink), just like google uses a lot of its TPU capacity for its own purposes.

dn762• 2 years ago

And yet Amazon and Azure have more market share in Cloud. Whatsapp in chat. Microsoft in Office. Zoom in vid conf. The Telcos in fibre, Netflix and Disney more than Youtube etc etc. Google's enemies are internal ad tech managers, who aren't interested in any other team overshadowing them. If Android is better than iOS it wont be used to take out Apple. It will be used to sell more Ads. Their Adtech guys run the show.

shikon7• 2 years ago

If they have 10x cheaper compute, and a lot of cash to spare, why isn't Google dominating AI, like SpaceX is dominating the launch of satellites?

Does Google not want to, or is there another reason they cannot?

mmaunder6• 2 years ago

There seems to be a consensus that we haven't yet seen the killer AI app that produces ROI that justifies the level of investment we're seeing. If that's true, then it's all still up for grabs.

My own view is that what AI is going after i.e. the size of the pie, is the $50 trillion per year portion of global GDP that makes up wages. Putting aside the risks - which are enormous - when you see it that way you begin to understand openai's valuation late last year, and that that's just the tip of the iceberg of what's coming.

The risk of disrupting a sizable chunk of the world's wages is fuel for a separate post and conversation. IMHO it's the real risk of AI, and the Skynet scenario around ASI/AGI that has been popularized is a distraction.

janalsncm5• 2 years ago

None of that matters if you are selling shovels. Cash is cash. Google could be selling TPUs and directly competing with Nvidia. You might think Nvidia’s valuation is inflated, but the fact of the matter is they sold $60B in hardware last year.

mullingitover• 2 years ago

My own view is that what AI is going after i.e. the size of the pie, is the $50 trillion per year portion of global GDP that makes up wages.

This is why AI is the smart investment play in the near term, but those ahead of the curve are long on canned goods and firearms.

dehrmann• 2 years ago

Not only isn't there a killer app, LLMs are getting to be commoditized.

raxxor• 2 years ago

I believe the killer feature would be sensible embedding tools with respective functions that support generating facts about internal data.

You generally cannot just push your data tables with millions of entries to your AI, but you can reduce them to statements that the AI then can work with.

It would be quite nice for training if a company can embed their product catalogue or their process instructions. Or technical data and common faults for customer support.

Also production monitoring would benefit from AI, but you also need to reduce the large amount of data into congestible statements. Usually the same work you do when you build some kind of dashboard.

In most cases that would mean you either run an AI locally, which I would heavily prefer, or you serialize the result of said embeddings and have them be used with your cloud AI of choice.

janalsncm5• 2 years ago

If you want to use an Nvidia GPU you can buy them and put them in whatever you want.

If you want to use a TPU you have to use Google cloud.

moralestapia2• 2 years ago

Why would someone want a TPU?

Not just that but, why would someone (not working at Google) want a TPU so much to put up with all the hurdles, including learning a completely new and non-portable programming stack?

moralestapia2• 2 years ago

or is there another reason they cannot?

I'll never get tired of saying this, extremely incompetent management starting from Shundar (or however that is spelled).

I'm quite sure they still have the best talent in the world, but they also have the kind of PM that does those dumb "my day as a PM in San Francisco" stories at the helm, this is the result, maybe a trillion USD worth of value is unrealized because of this.

krisoft2• 2 years ago

Shundar (or however that is spelled).

You could perhaps google it? And then you would learn that his name is spelled Sundar.

Also, “that” is not how one refers to human beings or their names. But at least we can know how much to trust your opinion. If you got tired postulating about their management but couldn’t be bothered to ever check the name of their CEO.

dekhn2• 2 years ago

The classic answer to any question like this at Google was "but does it have the same level of profitability as ads?" This was commonly cited as a reason Google shouldn't get into several different areas (cloud being one). I kid you not, one of the most common arguments I had when I joined google was that google was late to the cloud, the response was always "we have a cloud, it's called borg" and "we have a cloud, it's called appengine" while I'd have to explain to people that there was an entire market of people who wanted to "import numpy" on 128-core machines.

Another answer is that Google has quietly dominated industrial AI for some time; few people talk about SmartASS, or Rephil or SETI , and most modern ML folks would barely recognize Sibyl, yet those products help Google grow Ads into a monster business. YouTube (Watch Next) and Android (Play Store) both have Sibyl to thank for rapid growth at a critical time. Google played a big role in bringing around modern deep learning (voice recognition, language modelling) and is one of the largest, if the not the largest, industrial deep learning research publishers.

Another answer is that while google vends access to TPU thru cloud, it's a tricky product to maintain and sell for external users, and to keep the product affordable, they are making some very serious decisions about how much gets used internally vs. deployed (an Ads job running on a TPU is likely more valuable than a customer job running on a TPU) with massive capital expenditures on hardware and facilities that affect their earnings and profits.

Yet another answer is, Google is one of the largest resellers of access to nvidia GPUs but they don't dominate because they entered this business a bit late and their competitors (amazon and microsoft) are excellent at selling products to enterprise and at buying large piles of GPUs and running facilities.

I just don't think it's in Google's DNA to be highly profitable outside of a few businesses- a search page for ads, ads in videos, ads in applications, ads around the internet, etc, and it's unlikely to change any time soon, because their top leadership doesn't know how to cultivate new products (stadia and google+ being two particularly egregious examples).

ra7• 2 years ago

I just don't think it's in Google's DNA to be highly profitable outside of a few businesses

I've often wondered if this was the reason why the entity Alphabet was created, with X projects to "graduating" into standalone companies. Because otherwise most of them would die an unceremonious death from not being as profitable as Ads.

thehappypm• 2 years ago

The long game isn’t done yet, and “dominating AI” doesn’t mean “most popular chatbot in January 2025”

fragmede• 2 years ago

Contrary to the title, fhe post is comparing Starlink's access to cheap launches, not SpaceX launch capability vs the rest of the market.

What Google has done by vertically integrating the hardware is strategically similar to SpaceX’s Starlink, with vertically integrated launch capability. It’s impossible for any other space based ISP to compete with Starlink because they will always be able to deploy their infrastructure cheaper. Want to launch a satellite based ISP? SpaceX launched the majority of the global space payload last year, so guess who you’re going to be paying? Your competition.

aaomidi• 2 years ago

If people aren’t getting paid, then goods aren’t being bought and GDP will crash.

I don’t know what the end goal here is. Any sizable impact on wages will directly impact every other business sector.

morkalork• 2 years ago

Apple will sell phones to the few remaining Tesla employees, Tesla will sell some cars to the few remaining Apple employees, and the rest us will just eat each other?

jpetso• 2 years ago

There is no end goal. It's just a bunch of independent actors who are all trying to get ahead of their competitors a step at a time. Two steps if they're lucky.

The collective system steers into whichever direction was determined by the previous set of steps, with very little potential for any individual actor to turn it around.

throwawaymaths• 2 years ago

does google know how to sell hardware to hyperscalers?

mike_hearn• 2 years ago

The 10x figure is made up. We don't know if or by how much the TPU ecosystem is cheaper than Nvidia.

Firstly, the h100 margin number appears to be looking purely at the manufacturing cost vs sale price. Where are amortised driver, software stack and design costs in that? Nvidia has won because of the strength of the CUDA stack, including their uniform driver codebase which isn't free to develop.

The cost of a tpu or GPU isn't simply in the hardware costs. Google has to maintain an entire parallel software ecosystem as well. Then we can assume Nvidia is spreading design costs over many clients including gamers and the console industry as well, whereas Google isn't.

It might well still be a saving for Google because Nvidia is making bank right now, but if it was actually easy to replace these chips with in house designs and software stacks then all the big cloud companies would be doing it. They're trying, but so far those projects haven't worked out.

janalsncm5• 2 years ago

The author is right about the vertical integration. It’s just that none of the stages are great. I don’t use Google cloud and have no interest in learning its idiosyncrasies. And I have PTSD from Tensorflow. (It’s only a bit of an exaggeration to say no one outside of Google uses it anymore.) Using TPU in PyTorch is possible though.

I wish Google would compete with Nvidia directly and let me buy a TPU (the big boy ones, not the flash drive sized Corals). Because as it stands now, Google could have Nvidia’s business and OpenAI’s.

kajecounterhack3• 2 years ago

It’s just that none of the stages are great.

Really? Nobody uses TF but plenty of people use XLA, which is what you allude to saying that TPU in Pytorch is possible. That's arguably the most important piece; the sledgehammer that makes good performance and good devex compatible.

I wish Google would compete with Nvidia directly and let me buy a TPU

I don't see the utility of buying _any_ GPUs. It's mainly a cost + availability optimization to own them yourself if you're doing a lot of continual training. Outside of the foundation model companies, most of us just use cloud services -- and I want those to be cheap and always use the latest thing.

Google has a highly optimized infrastructure to make sure unused resources (CPU, TPU, RAM, disk) are properly allocated. This means preemptible instances, highly co-located compute + data, etc. In theory everyone should win when you use ML hw through hyperscalers, because they collect on their structural advantages and your TCO is lower.

Google could have Nvidia’s business and OpenAI’s

They arguably have a better _business_ on their hands but a worse _speculative outlook_ in the eyes of Mr. Market. Those are different things. There's an argument to be made that Google's valuation could be as high if they ran their public relations strategy as well as OpenAI / Nvidia / Tesla, which are all riding monumental hype.

janalsncm5• 2 years ago

most of us just use cloud services

Yes, but because cloud services buy GPUs almost exclusively Nvidia, it would likely drive cloud prices down as well.

motohagiography2• 2 years ago

if AI is a weapon, google wins, but if AI is a product, apple wins. google can build compute capacity, but solving cringe at their products isn't a compute problem.

products are hit songs, who here is going to write them.

askafriend2• 2 years ago

Since it's both, both companies will win big.

motohagiography2• 2 years ago

the "why not both?" worldview is kind of muddying and neutralizing to conversation tho. really, one model will prevail.

FabHK• 2 years ago

Minor nitpick:

NVidia’s margin on the H100 is 1000%. That means they’re selling it for 10X what it costs to produce

If the margin is 1000%, they're selling it at 11x what it costs to produce. Were they selling it at 10x, their margin would be 900%.

mmaunder6• 2 years ago

Don’t you just love percentages? Thanks.

chillee2• 2 years ago

One of the big things this article misses is that Google pays Broadcom a significant amount for the actual chip design, also around a 70% margin.

Google certainly has infra/cost advantages, but it's nowhere near 10x.

mmaunder6• 2 years ago

Mind sharing your source on that? I’ve been trying to find one.

Edit: Specifically the nature and current status of the Broadcom/Google relationship as it relates to TPUs.

chillee2• 2 years ago

https://www.theinformation.com/articles/to-reduce-ai-costs-g...

Which takes it from

Broadcom generates a 70% profit margin from its work on TPUs, said a person with direct knowledge of the internal analysis. SemiAnalysis, a chip research firm, earlier reported that figure.

https://semianalysis.com/2023/08/30/broadcoms-google-tpu-rev...

zoratu• 2 years ago

Interesting allusion to Desiderata, “Go placidly amid the noise and the haste, and remember what peace there may be in silence.”

Amidst the noise and haste, Google has successfully pulled a SpaceX

Related Stories