Is AI eating all the energy?

https://blog.giovanh.com/blog/2024/08/18/is-ai-eating-all-the-energy-part-1-of-2/

by lr0 • 2 years ago

29 62 2 years ago

blog.giovanh.com

monero-xmr2• 2 years ago

Energy use is only bad if the user is doing something you disagree with. When crypto miners were in the news, it was terrible because crypto mining "provides no value" (to the people doing the criticizing).

Now that AI is using massive energy and the usual nags are coming out to criticize it, suddenly HN is promulgating the glorious benefits of such excess energy use. Pot meet kettle. Human nature never changes

whatever1• 2 years ago

At least some of cpu cycles in llms result to useful computation. That is in stark contrast to crypto mining where computation is done for the sake of computation scarcity.

monero-xmr2• 2 years ago

Well if you truly believe Bitcoin provides safety against money printing, then the CPU cycles aren’t “wasted” but extremely valuable.

Luckily we have a market for energy and we can all decide for ourselves what is useful energy use

batch12• 2 years ago

I seem to hear less about reducing carbon footprints nowadays, but maybe it's just being drowned out by ai hype.

rembicilious• 2 years ago

You trigger an idea of AI evolution in my brain. AI needs power to proliferate. AI that encouraged power consumption and de-emphasised climate issues naturally thrived. Like a successful virus; no thought required. Survival of the fittest.

simonw• 2 years ago

There's a crucial difference between AI energy usage and Bitcoin proof-of-work energy usage.

Proof-of-work is set up as a competition. To earn coins, you need to burn more energy than anyone else (in order to win the hashing lottery). This means that energy usage trends upwards - if someone else builds a bigger coin farm than you, your incentive is to increase the size of yours even more.

With AI models, everyone is interested in finding and deploying new efficiencies. Have you noticed how the prices for hosted LLMs (OpenAI's gpt-4o-mini, Google's Gemini Flash, Anthropic's Haiku) keep dropping? That's because they keep finding new ways to optimize those models, serving useful results while using less energy to do it.

The AI space still has its competitive pressure: Google, Meta, Microsoft are all making huge speculative investments in GPUs and new data centers right now. But it's still not in the same class of waste as the proof-of-work competition used by Bitcoin.

wmf• 2 years ago

PoW is bad because PoS can do the same thing with 99% less energy yet some people refuse to switch. AI is already as efficient as we know how to make it.

Energy is good because it allows people to do things they care about. As long as the externalities are internalized [1] and the energy is paid for, the rest will mostly take care of itself.

[1] they aren't of course, but that's not crypto's fault and it's not AI's fault either

samstave• 2 years ago

I wrote up a thing on this:

HNer @externedguy "..built interactive map of active & decommissioned nuclear stations/reactors"

https://news.ycombinator.com/item?id=41189056

---

So I wrote the following in response:

(I correlated the Nuclear reactor locations with DataCenters, undersea cable endpints (which will be near both nukes and datacenters)

As they could be layers - and we track shipments and we can see where AI consumes:

---

...if we add the layers of the SubmarinCableMap [0] DataCenterMap [1] - and we begin to track shipments

And

https://i.imgur.com/zO0yz6J.png -- Left is nuke, top = cables, bottom = datacenters. I went to ImportYeti to look into the NVIDIA shipments: https://i.imgur.com/k9018EC.png

And you look at the suppliers that are coming from Taiwan, such as the water-coolers and power cables to sus out where they may be shipping to, https://i.imgur.com/B5iWFQ1.png -- but instead, it would be better to find shipping lables for datacenters that are receiving containers from Taiwain, and the same suppliers as NVIDIA for things such as power cables. While the free data is out of date on ImportYeti - it gives a good supply line idea for NVIDIA... with the goal to find out which datacenters that are getting such shipments, you can begin to measure the footprint of AI as it grows, and which nuke plants they are likely powered from.

Then, looking into whatever reporting one may access for the consumption/util of the nuke's capacity in various regions, we can estimate the power footprint of growing Global Compute.

DataCenterNews and all sorts of datasets are available - and now the ability to create this crawler/tracker is likely full implementable

https://i.imgur.com/gsM75dz.png https://i.imgur.com/a7nGGKh.png

[0] https://www.submarinecablemap.com/

[1] https://www.datacentermap.com/

----

And a while back I posted:

In the increasingly interconnected global economy, the reliance on Cloud Services raises questions about the national security implications of data centers. As these critical economic infrastructure sites, often strategically located underground, underwater, or in remote-cold locales, play a pivotal role, considerations arise regarding the role of military forces in safeguarding their security. While physical security measures and location obscurity provide some protection, the integration of AI into various aspects of daily life and the pervasive influence of cloud-based technologies on devices, as evident in CES GPT-enabled products, further accentuates the importance of these infrastructure sites.

Notably, instances such as the seizure of a college thesis mapping communication lines in the U.S. underscore the sensitivity of disclosing key communications infrastructure.

Companies like AWS, running data centers for the Department of Defense (DoD) and Intelligence Community (IC), demonstrate close collaboration between private entities and defense agencies. The question remains: are major cloud service providers actively involved in a national security strategy to protect the private internet infrastructure that underpins the global economy, or does the responsibility solely rest with individual companies?

readyplayernull2• 2 years ago

Isn't it disconcerting that brains consume so little energy doing continuous high complexity tasks like visual processing and even resting produce vivid simulations, while numeric calculations require so much effort for advanced brains? It's like the energy usage pattern is completely opposite between brains and computers. That should tell something about our approach to intelligence by using, maybe, the wrong tool?

drdaeman• 2 years ago

brains consume so little energy doing continuous high complexity tasks

Do they? While I've read that there's not a lot of difference between just chilling and thinking of some hard problem (not sure if true, though - haven't really dug to verify), thinking is thinking, so I guess that makes sense.

But when e.g. a reaction time is important, I suspect, it could be quite energy-hungry. I mean, my watch sends me that I have elevated heart rate when I'm playing any fast-paced videogames, and I guess all that blood pumping surely has some biological reason (supplying more oxygen or something) and isn't just for funsies. I'm not a biologist, so this is purely based on my own observations and stuff I've heard somewhere.

eric-hu• 2 years ago

While I've read that there's not a lot of difference between just chilling and thinking of some hard problem

I find that difficult to reconcile with chess grandmaster calorie consumption.

https://www.npr.org/2019/09/18/762046422/the-chess-grandmast...

Polard, this company that tracks heart rates, monitored chess players during a tournament and found out that this 21-year-old Russian grandmaster, Mikhail Antipov, had burned 560 calories in two hours, which we found out was roughly what Roger Federer would burn in one hour of singles tennis.

interstice• 2 years ago

Degrees of entropy perhaps? CPU's are low entropy in that they are highly ordered and organised - firing only when specifically expected to, whereas brains are high entropy in that they get to a result by riding a fine line between randomness and order and those connections are not inherently expensive.

That and billions of years of optimising for low energy switches.

readyplayernull2• 2 years ago

Yeah, there could be something. Computers fight entropy to gain precision, so intelligence should benefit of fuzzyness to get the "free energy meal". But that brings a sort of uncertainty principle. We might be able to grasp what intelligence is, but like a cloud of electrons we will never be sure of its detailed state.

cen4• 2 years ago

Babies don't come installed with those brains. They take a decade or two to pick up most skills that have any value. And after all that, the vast majority end up playing replaceable support npc roles. What's the energy consumption? To get from - why is the compass pointing north to electromagnetism, it took thousands of years and the activity of millions of brains. We see a highly misleading and compressed version of that computing process in our textbooks. As they say knowledge is inherited wealth. People over estimate how much they deserve it or have earned it.

visarga3• 2 years ago

A child learns to talk in about 150M tokens, GPT-4 needs 15T tokens. Humanity needed the total speech of 110B people to evolve to current level, which comes about 51,395,437,500,000,000,000 tokens, about 3.4 million times more than GPT-4's training run, and 342M times more than a child needs to learn.

That is to show the relative cost between imitation and discovery. Evolution is on the order of 0.3 trillion times slower than catching up by imitation.

achierius• 2 years ago

support npc roles

This might sound trite, but are your parents "replaceable"? Even in a strict mechanistic sense, even a "baseline" parent possesses innumerable qualities that we have absolutely no clue how to build in a robot. They have a well-tuned cost function that keeps them or anything else from killing the baby; they come to learn the child well enough to understand what they're thinking, to correct them & guide them; they themselves are reliable enough to broadly survive until the child reaches adulthood (can you imagine swapping out your robot parents every iPhone generation?). Even their fine motor skills alone are far beyond what we today understand how to build (to cradle, feed, wash, &c).

And that's not the only thing, just a convenient example of how the worldview you present -- one where the only value someone can provide is that which can be extracted by the owning class via their "job" -- is not only unkind but also incoherent. I don't even know what you mean by "deserve" or "earned": yes, none of us did anything to gain the attributes that we were born with, but so what? We are who we are, and who we are is defined in scope by who we were born to be. We are all given options and choose between them day by day. But nobody -- no thing -- chooses which options they are given.

Retric• 2 years ago

We don’t hyper optimize computers for energy use the way evolution did. But even still the brain uses about as much energy as 4 Apple M3 chips which isn’t nothing in terms of computer AI. Multiply that by years of training time and it doesn’t seem that crazy different IMO.

trilbyglens• 2 years ago

I think more impressive is that the energy supply system is efficient enough to run a days worth of software generation off of a ham sandwich.

jordanb• 2 years ago

Whenever anyone points out that what LLMs are doing is definitely not how a brain works the AI people respond with "maybe it's going to be better!"

teaearlgraycold• 2 years ago

LLMs are reversing language from text. Language is as much an extension of the brain as the retina is. The words are arbitrary but the deeper levels are closer to rubbing your neurons on a listener’s ear drum. Why would a brute force simulation be anywhere as efficient as the original hardware?

caseyy3• 2 years ago

You also have to consider what intelligence is useful. We already have human brains in abundance, but computers can do certain things we can’t.

I think LLMs are a bit of a red herring. We will discover powerful ways to use machine learning that are very different from human brains. The goal doesn’t need to be to replace them.

planckscnst• 2 years ago

Computers aren't evolving in an energy-scarce environment (yet) we (currently) pretty much give computers as much energy as they want. Computers are evolving in an environment where a dependent species relies on them for that species' own weaknesses.

NoPicklez• 2 years ago

Well the human approach to intelligence and its tool has had approx 6 million years to improve and evolve and has created the reality we experience today, which could be really intelligent behavior or not. We don't have anything to compare it to.

Think of the size of the computers we had in the 1960's and their computing power, now you have that multiple times over on your wrist. The tools we use seem to get bigger and bigger, then smaller, then bigger until a new paradigm becomes available (e.g. quantum)

burnished3• 2 years ago

I misread your statement as stating that computers broadly were less energy efficient at the same tasks, because I took umbridge at your claim that human brains use `so little energy` (citation needed???) and forgot how to reading comprehension, so I compared training costs of a 7 year old and inference costs for an adult in terms of energy to the figures quoted in the article for the various models. I'll address what you actually said below but am keeping my original writeup because its kind of neat.

I'm going to compare energy costs for training a model vs growing a brain because they are analogous. TFA lists the training costs in kWh and I'll arrive at a comparable figure by estimating the total caloric intake of a 7 year old and then multiplying that by an estimate of the proportion of energy a child's brain uses.

I calculate that the lifetime caloric intake of a 7 year old is 3,139,000 dietary calroies based on a daily schedule of 1,000 for the first three years and 1,400 for the next four[1], and assuming an active child's brain uses roughly 60% of total calories[2], we end up with a lifetime total of 1,883,400 calories spent on the brain. This is 2,188 kWh.

> the energy consumption of model training is somewhere in the range of 11,000 kWh to 52,000 kWh for moderate-sized LLMs, 40,000 kWh to 60,000 kWh for more expensive image generators, and up to 5,000,000 kWh for global-scale LLMs like Llama and ChatGPT.

The 7 year old isn't even finished training, and frankly, needing the same training energy as 2,285 7 year olds to create ChatGPT comes across as a bargain.

Just for a bonus lets compare inference costs, assuming a human brain uses roughly 200 dietary calories and that we can count all of them as part of performing productive work gives us a budget of about 0.5 kWh (464 Wh). Going to quote the article again for inference costs

> the energy consumption of an inference operation is somewhere in the range of 1.0 Wh to 7.0 Wh for standard operations, and to 4 Wh to 7.5 Wh for more expensive jobs like image or video generation.

Hundreds of text operations and 50-100 images.

As a back of the napkin sketch it looks pretty close.

[1] https://www.healthychildren.org/English/healthy-living/nutri... [2] https://www.livescience.com/burn-calories-brain.html

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You're sorta onto something but it has to do with architecture: human brains are composed of a vast number of sparsely connected and individually poor computational units (neurons). Computers are completely connected and highly reliable but individual units are expensive. Neurons also communicate at a much slower pace than computers can perform calculations. The reason for our unreasonable effectiveness appears rooted in how very many neurons we have and how effectively parallelized most functions of the brain are. Precision calculations are easy for computers because they are literally machines that have operations that map to simple numerical calculations that are highly accurate. Numeric calculations are not simple for human brains because nothing really maps to the process of a calculation. Individual neurons do 'integrate and fire' meaning that if they get excited enough by nearby neurons in a short enough time then they will fire and excite the neurons they touch, but it isn't a reliable process, meaning that if neuron A touches neuron B and A activating is enough to cause B to activate this time it may not be enough the next time A activate. So when we do math we use those.

There are investigations into neuromorphic hardware but I'm not up to date on them anymore, the basic idea is to make computing hardware inspired by how the brain operates. Last I knew there were some interesting results but nothing resembling anything practical.

axdsk• 2 years ago

The right tool might be Spiking Neural Networks. Due to their sparse activation, event driven computation, and temporal coding. This all depends on how good neuromorphic chips get.

pants2• 2 years ago

Bitcoin only uses huge amounts of energy because the bitcoin community likes it that way. Nearly all other blockchains use some form of Proof of Stake model and their energy use isn't much more than a typical cloud service.

toenail• 2 years ago

And all proof of stake blockchains require trust in some entity. Proof of work solves that problem, it's not just a matter of preference.

ben_w• 2 years ago

PoW creates the illusion of solving trust, but it is only an illusion because all things one may purchase with a transaction also require trust.

At best, there's also smart contracts, which do have real uses, but they also have limited domain — you can't meaningfully digitally verify the correct delivery of a pizza, or that the pizza is correct as ordered, or that it's even safe to eat.

Legal system does all that, but if you've already got a legal system then you don't need to worry too much about trusting the bank either.

mjfl• 2 years ago

AI is not useful enough to use all this energy. Sounds like capital destruction.

synicalx3• 2 years ago

I've often wondered if we're focusing on the wrong things with lines of thinking like "does X use too much power?". It's not like our demand for electricity is ever going to be lower than it is right now, so trying to use less of it seems futile. We can generate immense quantities of electricity given enough time and money, we've obviously fallen well behind if electricity is considered a limiting factor.

Disclaimer; I'm not talking specifically about AI here, just production and consumption of electricity in all it's forms.

alanbernstein• 2 years ago

But if the growth rate of usage in a certain sector seems to outpace everything else, that can give us clues about how those trends may have to change in the future.

synicalx3• 2 years ago

From a capacity planning perspective that's very useful information for sure.

totallymike• 2 years ago

Do you ever wonder where that electricity comes from? It might be worth checking out if you’re not convinced that finding ways to stay efficient in the face of enormous technological growth is worthwhile.

synicalx3• 2 years ago

I'm not really talking about the electricity we (global 'we') are generating right now, because we're still talking about curtailing our usage of it so it's obviously not enough. I'm talking more about electricity we could be generating in future so that demand is met - that should, and can be from clean sources like solar, nuclear, hydro, geothermal, wind etc.

Efficiency should be an implicit goal with everything, but that's only ever going to blunt the demand not stop it entirely. We still need massive growth in generation in the coming decades, or we have to accept that the pace of everything is going to slow down.

insane_dreamer2• 2 years ago

our demand for electricity is ever going to be lower than it is right now, so trying to use less of it seems futile

not necessarily; for example, switching from incandescent lightbulbs to LEDs is a huge decrease in power consumption

_heimdall4• 2 years ago

In isolation, yes one led bulb is more efficient. People tend to use that efficiency as an excuse for using more on the whole though.

I wish I could find the link here, without it this is just anecdote, but I saw a study that found our energy use for lighting either remained flat or increased with the move away from incandescents. We install more lights and feel better about leaving them on for longer periods of time.

jillesvangurp2• 2 years ago

We live in interesting times. Data centers are starting to use a lot of energy. But in the grand scheme of the energy market rapidly trending towards using renewable energy, that's actually OK.

Energy usage is bad for our planet when you use things like coal and gas plants to generate the energy. But the vast majority of installed new capacity is actually a mix of wind and solar supplemented with battery. That mix is growing at a rate measured in TW/year of added capacity. It's well over 80% now.

A lot of new gas and coal capacity is increasingly reserved for use rather than actually used. Shiny new gas plants designed to run 24x7 are actually now being pushed into a role as peaker plants. Which means they are rapidly becoming financial basket cases. Some relatively new plants have actually been closed already for this reason.

More coal gets decommissioned than built at this point (China and India are still adding capacity, the rest of the world mostly isn't). For the world, peak coal was last decade. And even in China, peak coal seems near.

In terms of overall fossil fuel usage, things are about to peak and then decline in terms of fossil resources used. What that means is that from now on the amount of fossil fuels (coal, gas, oil) used for electricity production should start to decline not just in relative terms but in absolute terms.

Don't get me wrong, it's still pretty bad and short term the decline is maybe not yet quite happening or that easy to observe. But in a few years there should be some measurable steady decline. Experts seem to disagree whether that's right now or by something like 2028. But the point is that we're at or close to the peak here.

Cloud computing and AI are one of the big drivers for all this. They create demand for more energy. This causes the energy market to grow. But as it grows, fossil fuel usage is starting to shrink. That's a trend that accelerates as renewable energy and batteries get cheaper (learning effects, technology improvements, etc.).

There are some pretty interesting economics at play here that boil down that doing more of a thing causes that thing to become cheaper. Not just a little bit but actually by quite a lot. So, there's value in creating new demand: it speeds all this up.

Cost per mwh is an interesting number to track. That has been dropping steadily over the last decade and a half. Anyone using a lot of energy, e.g. data centers, is going to be looking for the most cost effective way to source that energy. And renewables are at this point the cheapest source by far.

The main premise of this article seems to suggest that we should slow down things like AI because it uses too much energy. But by creating a need to grow the energy market, we're actually speeding up the demise of fossil fuel generation; not slowing it down. So, I'm not so convinced that putting a lot of energy (pun intended) into attempting to slow this down is that productive. It's probably futile and the effects are debatable (I would argue negative even).

Besides, computing resources are enabling a few things that are actually causing behavioral changes. For example, we travel a lot less for work. And we can work remotely. Also a lot of work is getting automated entirely removing the need to move humans around to do the work. AI is projected to be a big part of that. It has an energy cost. But it also has upsides. And that cost can met sustainably. So, it's not all that bad.

oezi• 2 years ago

The overprovisioning of solar is going to create so much cheap surplus energy during summer and during the day that it will be very interesting which applications will come up to consume it.

swamp_donkey2• 2 years ago

Who will pay to build these solar farms whose output is worthless?

_heimdall4• 2 years ago

The challenge there is always going to be the energy use required to create and maintain these renewable systems, including energy storage.

The great lie of green energy, in my opinion, is that we can invent what is functionally a perpetual motion machine. There are always costs and losses in the system, even when they are only up front and at the end of life.

lokimedes• 2 years ago

Progress (empowerment) always costs more energy, that is thermodynamics. We may be surprised in what form the expended energy takes, or skeptical about the benefits (think whale oil lamps), but that is the general pattern.

That we (and especially eco-dreamers in Europe) saw the stagnating energy consumption per capita over the last 3 decades as evidence that we can save energy to reduce climate change is a fiction that can only be held by people who haven’t witnessed true progress (or cared for it).

_heimdall4• 2 years ago

I don't that its fair to assume that progress alwaysalqays requires more energy.

Many innovations that we would consider progress offer more efficiency than what they replace. The challenge is that we continue to consume as much as possible. New innovations often have the potential to decrease energy use through efficiency, instead we choose to use more of the thing.

I.e. "progress" often offers a decrease energy consumption per unit, but we end up with a total energy use increase when we drastically increase the unit count.

zekrioca• 2 years ago

I think the comparisons to images are a bit misleading, where the author implies that generating images are somehow more energy-efficient than taking the photos yourself.

It basically ignores the fact that the very data these large models use came from real photos taken from real cameras etc. and that used real energy, which in the end should be included in the total training cost, i.e., in addition to the electricity consumed by the GPUs. This would have an impact in how the total cost can be amortised with usage. Of course the impact might be just to slow down a little the amortisation with use, but it still should be included for a fair comparison.

Same applies to some of the other examples they give.

1vuio0pswjnm7• 2 years ago

"Recent tech trends have followed a pattern of being huge society-disrupting systems that people don't actually want."

"While planned obsolescence means this applies to consumer products in general, the recent major tech fad hypes - cryptocurrency, "the metaverse", artificial intelligence... - all seem to be comically expensive boondoggles that only really benefit the salesmen."

But acccording to sentence #1, some people do want these things, e.g., salespeople, investors, etc. But as sentence #2 implies, this is not a majority of people. Over the years, when people complain on HN about software and the web, other commenters, presumably software/web developers, respond that these complainers do not matter because they are only a minority: the majority of people do not complain. Failure to complain apparently means the silent majority "wants" what they get.

What rule can we derive from this developer "reasoning/argument". Sometimes it makes sense to cater to a minority, other times it doesn't?

"If you're not familiar, data centers are dedicated facilities for running servers. Data centers are "the cloud": instead of running your own servers, you can rent computer power from experts who are very good at keeping computers from turning off."

"The problem we immediately run into if we try to think about the proportional cost of AI is that there is no consensus on whether it's ultimately useful."

I think it is safe to say that only a minority of people are speculating that "AI" is useful. The majority are silent. Does that mean the majority want it.

We know what salespeople, investors and developers will say.

"As I mentioned earlier, the AI boom feels a lot like the blockchain cryptocurrency push of a few years ago. Like cryptocurrency, AI is a tech fad, it requires data centers, it consumes more energy than a webserver... the comparison is extremely natural."

If these things were truly useful, then it stands to reason the majority of people would be asking for them. Instead we see indifference. To developers, this indifference equates to "Yes, we want it".

A fundamental problem with paying atteniton to the speculation of developers and other pundits who advocate expensive, unnecessary uses of computers, e.g., "AI", besides the obvious conflict of interest, is that these folks do not have a good track record of being honest. To start MicroSoft in the 1970s Bill Gates had to lie that he had software that did not yet exist. This "vapourware" tactic might seem quaint but this culture of dishonesty now involves much higher stakes.

Developers are still faking demos. Putting Elizabeth Holmes in prison has not stopped the Silicon Valley culture of fraud.

Is AI eating all the energy?

Related Stories