Environmental Cost of AI

Think about the last time you asked a chatbot to write a quick email or summarize a document. It felt instantaneous and free. But behind that simple interaction, a massive physical infrastructure was at work, consuming electricity and generating heat. Here is the key insight: AI is not a cloud that floats in the sky; it is a physical industry with a rapidly growing environmental footprint.

To lead responsibly, we must understand the physical cost of our digital decisions. Training a single model like GPT-3 generated an estimated 552 tons of CO2—roughly equivalent to driving a car 120 times around the Earth. And that is just the "tuition" cost to teach the model. The ongoing cost of using it is even higher.

You might assume that training the model is the biggest environmental offender because it requires thousands of GPUs running for months. While training is intense, it happens once. Think of it this way: Training is like manufacturing a car, while "inference" is driving it every day.

Inference is the process of using the model to make predictions or generate text. For popular models, inference consumes more energyinference consumes more energy in the long run than training. Meta, for example, has reported that the vast majority of their AI infrastructure energy goes toward serving users, not training models. Every query counts.

Energy is only half the story. High-performance chips generate immense heat, and data centers often use water to keep them cool. It is estimated that a simple conversation with a chatbot (roughly 20 to 50 questions) consumes about 500ml of water. In regions facing drought, this water usage creates a direct conflict between technological progress and local resource needs.

When we assess the environmental impactsenvironmental impacts of our systems, we must look beyond the electricity bill. We need to consider the local water stress in the regions where our compute providers operate.

You might be wondering: "Can't we just use renewable energy?" Yes, and location is your most powerful lever. The carbon intensity of the power grid varies wildly. Training a model in a region powered by coal can result in carbon emissions ten times higher than training the exact same model in a region powered by hydroelectricity.

As a governance leader, you can make a significant impact simply by asking your cloud providers where your workloads are running. Moving non-urgent training runs to low-carbon regions is a high-impact, low-effort sustainability strategy.

Sustainability shouldn't be an afterthought. It should be part of your Organizational AI Compute Governance ChecklistOrganizational AI Compute Governance Checklist. We need to treat compute and energy as finite resources to be managed, just like budget.

I recommend implementing a Sustainable AI Reporting FrameworkSustainable AI Reporting Framework to track your metrics. Are you using a massive foundation model for a simple classification task? Could a smaller, more efficient model do the job? By right-sizing your models and optimizing your infrastructure, you save money and reduce your organization's footprint. The EU AI ActEU AI Act and other emerging regulations are beginning to mandate this kind of reporting, so building these muscles now is a strategic advantage.