The Hidden Power Cost of AI: Why Data Centers Need 40% Energy Just for Cooling

Hello, and welcome to today's AI Briefing. My name is Tom. Uh, for anyone who's new to the AI Bri- we, uh, we come up with short news snippets, information, stuff that helps people cut through the, uh, noise in the area to try and understand what's actually going on in the AI environment, 'cause there's always a lot going on. And so today, we're in a rather chilly, rather snowy Washington, D. C.

, and we're going to talk about AI cooling demands in the, uh, data centers that are required to drive all of the [chuckles] power and the stuff that needs to get done to allow you to ask ChatGPT your favorite questions. So, over the course of the last few years, of course, the usage of data, data centers, and compute to be able to drive LLMs and everything that goes with it has increased dramatically. S- say that, uh, data centers will use two to four percent of global electricity in the near term, which of course is a substantial amount when you think about it just powering computers. Now, as a society, we make a lot of use of computers, but two to four percent is, is a large number, and of course, it's going up with the amount of processing that's required to be able to facilitate all these ChatGPT, Claude, whatever, LLMs. And so AI, of course, is causing a spike of all of this because the requirements for people to get real-time access to huge corpuses of data, ask all these questions, and actually find out what's going on and get the sensible results, which is the big problem when it comes to AI and trying to get, um, some sensible answers out of all of this.

The electricity consumption, though, isn't just compute, of course. It's also the cooling that is required to drive the systems themselves. So according to a recent study, forty percent of data center power went on compute-ish, and then thirty-eight to forty percent of data center power then went on the cooling systems to actually be able to cool it. Now, there's obviously snow on the ground here, for anyone who's watching the YouTube video. For anyone who is [chuckles] listening to the podcast, hopefully you can hear the snow under my feet.

And of course, forty percent of the electricity consumption just to cool the systems is a big deal. So there's many different, uh, programs underway where companies and organizations are trying to find ways to be able to cool the data centers and the compute within the data centers in different ways. And so, for example, it's not snow related, but Microsoft in recent years have put, uh, data centers in the water. I know they're not the only organization to do so, but they're more, more prevalent, and they sunk a bunch of compute into the ocean. Now, of course, that sounds great, but the problem is if you did that on a huge scale, then you would also potentially warm the ocean that's around the compute.

Other countries like Finland are working on burying their data centers deep underground in mines, repurposed mines, that type of stuff, where you've got access to a very cold climate and very cold bedrock that can also help drive the cooling. Of course, the offset of that is that you then end up w- warming the earth that surrounds them. Now, is that gonna be so much that it causes, um, you know, permafrost melting, what have you? I don't know. But, um, if you did it at a large enough scale, of course, the heat has to go somewhere.

So I guess the point of this podcast, just briefly today, is just to talk through that and understand that everything has a knock-on impact. And the more that we rely on LLMs and the more that we make use of them, until more efficient compute chips come into play, the power usage and the consumption of cooling to be able to deal with that and get that heat out of the computers and into the environment will forever increase. And over the course of the next few years, I suspect an awful lot of different organizations and industries will start to look at that and try and figure out more, you know, start to expand on these cooling programs and different ways of being able to deal with that. Chip efficiency, though, of course, will end up being the key driver of this. If you can make the chips super efficient and not leverage an awful lot or not produce as much heat as they currently do, then that will end up being a much more effective way of being able to control the amount of heat and cost of electricity that's used to then cool the data centers.

And so of course, the knock-on impact of this as well is to the consumer, because if you can then spend forty percent less on your electricity bill, entirely forty percent, but say you knock twenty percent off your electricity bill because your chips created half as much heat and needed half as much cooling, then that's a dramatic or, like, reduction in electricity cost to the data center, which would then go and impact both electricity prices on the wholesale market, but also the amount it would cost the data center to be able to run your compute models. Anyway, there we go. I figured I would use the snow as a example as to, you know, some of the ways that people are looking at cooling compute as the demands for it ever increase. And so I hope you found this useful. If you have, pop, uh, comments into the box below.

If you're looking online, you can always reach out to me at, uh, tom@conceptocloud. com. And in the meantime, I will leave you with that, and have a great rest of your day. Thanks for tuning in.