The Hidden Energy Bill of AI: What It Really Costs to Ask a Chatbot a Question

So, what did that question just cost?

Last week, one of us asked a chatbot to settle a pointless argument: does a hot dog count as a sandwich? The answer came back in about two seconds, nicely formatted.

Then a different question popped into our heads. Where did that answer actually come from? Not the words, but the power. Somewhere, a machine we’ll never see did a huge amount of math and got warm doing it. Someone paid for that electricity, and someone had to cool that machine down.

We write about AI and engineering on this blog, and honestly, we’d never stopped to think about it. So we went digging. Here’s what we found, explained the way we wish someone had explained it to us.

Two bills, not one

Think of an AI model like a restaurant. Before it serves a single meal, someone has to build the kitchen, hire the chefs, and test hundreds of recipes. That’s training. It happens once per model, takes weeks or months on thousands of chips, and costs a fortune in electricity.

Then the doors open and people start ordering. Every plate that leaves the kitchen is inference. One plate is cheap, but a popular restaurant serves a lot of plates.

That second part is the one most people underestimate. With hundreds of millions of people using chatbots every day, running the models now uses more energy over a model’s life than building it. Some estimates say inference overtakes training within about six months of a model going live.

So how much does one question use? Probably less than you’ve heard. For years, the internet repeated a figure of about 3 watt-hours per ChatGPT query. That number turned out to rest on old assumptions, including older and less efficient chips. Epoch AI redid the math and landed closer to 0.3 watt-hours, roughly ten times lower.

The companies themselves now report similar numbers. Google says a typical Gemini text prompt uses about 0.24 watt-hours, plus about a quarter of a milliliter of water, roughly five drops. OpenAI’s Sam Altman has put the average ChatGPT query at about 0.34 watt-hours.

To put 0.3 watt-hours in perspective, a 10-watt LED bulb burns through that in under two minutes. So no, your hot dog question didn’t melt a glacier.

But that’s only true for short, simple questions. Paste in a giant document and the cost can climb to around 40 watt-hours, more than 100 times higher. The International Energy Agency (IEA) has warned that video generation, reasoning, and agent-style tasks can use hundreds to thousands of times more energy than a basic text answer. The cheap question is cheap. The heavy stuff isn’t.

Where the electricity actually goes

A single question is tiny. The problem is that billions of them land in the same kind of building: the data center.

Picture a warehouse full of tall metal racks, each packed with servers and AI chips. The servers themselves take the biggest share of the power, around 60% of a modern data center’s electricity according to the IEA. The rest goes to storage, networking, and the thing nobody thinks about until it breaks: cooling.

Here’s the engineering reality that hit us hardest. Almost every watt a chip uses turns into heat. Pack enough AI chips into a rack and you’ve basically built a very expensive space heater. The IEA estimates a single advanced AI server rack could soon draw as much power as 65 homes.

Getting that heat out takes fans, chillers, and often water. Many data centers cool themselves by evaporating water, which is why you see headlines about AI and drought. Per query, it’s only a few drops. Across a whole region full of data centers, it adds up fast.

That’s also why location matters so much. Engineers look for cheap, reliable power from a grid that can handle a huge new customer, a cool climate so outside air can do some of the cooling for free, and access to water, or a design that doesn’t need much of it.

When too many data centers pile into one place, the local grid feels it. In Ireland, data centers already use about a fifth of the country’s electricity, and the IEA has estimated that share could reach 32%. That’s not just a tech story anymore. It’s a utility bill story.

How engineers are fighting back

This is the part we found exciting, because it’s where engineering gets creative. Demand isn’t slowing down. In 2025, data center electricity use grew 17% while global electricity demand grew just 3%, and the IEA expects data center use to double by 2030, with AI-specific demand tripling. So the question isn’t how to stop it. It’s how to do more with less.

The first fix is better chips. Each new generation of AI hardware does more math per watt, which is a big reason per-query energy dropped from around 3 watt-hours to around 0.3.

The second is liquid cooling. Air can’t carry heat away fast enough from today’s tightly packed racks. Running liquid right up to the chips moves heat far more efficiently, and some systems even reuse that heat to warm nearby buildings.

The third is smaller, smarter models. Not every question needs the biggest model on the planet. Sending simple requests to small, efficient models and saving the heavy ones for hard problems cuts a lot of waste. A UNESCO report found that small changes like this could reduce energy use by up to 90%.

The fourth is new power sources. Tech companies are signing deals for solar, wind, and even small nuclear reactors to power their data centers directly. The IEA counts around 45 gigawatts of small modular reactor agreements from data center operators.

None of these is a silver bullet. Together, though, they explain why a single question today costs a fraction of what it did a couple of years ago.

So, is it worth it?

We went back and forth on this one, and we don’t think there’s a clean answer.

The case for yes: a normal chatbot question uses about as much energy as running an LED bulb for a couple of minutes. If AI helps a doctor catch a disease earlier, helps an engineer design a more efficient building, or saves you an hour of work, that’s a pretty good trade. And efficiency keeps improving.

The case for slowing down: tiny numbers multiplied by billions stop being tiny. Data centers are now big enough to strain power grids and water supplies, and much of their power still comes from fossil fuels. And as AI gets cheaper per question, we tend to use it more, so total demand keeps climbing even as each question uses less energy.

Where we’ve landed: feeling guilty over a single question is probably misplaced. But the bigger picture deserves real attention, from the engineers designing these systems, the companies choosing where to build, and all of us deciding when we need heavy-duty AI and when a quick search will do.

Next time you ask a chatbot something silly (we’ll still be asking ours about hot dogs), remember there’s a whole building humming somewhere to answer you.

Now we want to hear from you: does knowing the energy cost change how you use AI? Tell us in the comments, or send us a DM on Instagram.

Sources: Epoch AI via TechCrunch, Google and OpenAI figures via Tech Insider, International Energy Agency “Key Questions on Energy and AI” (April 2026), and UNESCO’s 2025 report on LLM energy use.