By Chris Noble, CEO and co-founder of Cirrus Nexus
Since it burst into the public consciousness last fall, generative artificial intelligence (AI) has spurred many exciting advances and innovation, as well as serious dialogue about its implications for jobs spanning various industries. However, there has been much less conversation on the environmental impact of generative AI and how companies can use it responsibly and sustainably. The information technology industry is already estimated to account for two to four percent of total global greenhouse gas emissions – more than the aviation industry[1] – and is on track to require more energy for computing in 2040 than is produced today.[2]
AI consumes energy in two main ways: training and inference. Training is the process by which AI learns to identify patterns and relationships between data points. The more parameters the model uses, the more accurate its answers will likely be in the inference stage. However, training on larger datasets and/or with more parameters exponentially increases the computing power required.
The inference stage refers to the usage of the AI model after training to make predictions or generate content based on prompts. Each inference requires substantially less computing power, since the system has already set parameters and learned patterns. However, replying to a simple prompt usually requires several inferences. Doing this quickly enough to achieve desired outcomes can consume a large amount of computing resources, especially when a system is serving many users simultaneously.
When it comes to training generative AI models, the process consumes a staggering amount of computing power and energy – more than predictive AI technology. GPT-3, which ChatGPT is partly based on, was reported by Google and UC Berkeley researchers to have used an estimated 552 tCO2e in CO2 equivalent emissions or 1,287 MWh in energy consumption during training.[3] That’s as much electricity consumed as 121 U.S. households in an entire year[4]!
Similarly, Meta’s OPT-175B was developed with an estimated 75 tCO2e, but this doubles to roughly 150 tCO2e when including ablations, baselines, and downtime.[5] Meta researchers have reported that its AI training has grown to a 3.2x increase in data ingestion bandwidth demand from 2019-2021 and a 2.9x increase in training infrastructure capacity over 1.5 years.[6] Equally alarming are the results of a 2018 analysis by OpenAI, the makers of ChatGPT, which showed that since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.4-month doubling time. For comparison, Moore’s Law predicted that compute efficiency would double every two years.[7]
For inference, there is unfortunately even less data available on the energy consumption and environmental impact of generative AI. Recent research from Northeastern and MIT showed that inference has a substantially greater impact on energy consumption than training,[8] and AWS and Nvidia have estimated that inference can be up to 80-90% of total operational costs in deep learning.[9][10]
At Google, machine learning (ML) energy use across research, development and production was 10-15% of Google’s total energy use, in a study done across one week in April from 2019-2021. About 3/5 of Google’s ML energy use was for inference and 2/5 for training.[11] Similarly at Meta, inference was found to be anywhere from 50-65% of machine learning’s operational carbon footprint and increasing inference demands led to a 2.5x increase in inference infrastructure capacity from 2019-2021.[12]
While the exact numbers remain elusive, it is nevertheless clear that this boom in generative AI will only increase carbon emissions in IT. And that’s not even considering the impact on water, as UC Riverside and UT Arlington researchers have estimated that training GPT-3 could directly consume 700,000 liters of clean freshwater, and ChatGPT inference could consume a 500 mL bottle of water for a short conversation of 20-50 questions and answers.[13] So, what should companies do if they want to leverage generative AI without backtracking on their sustainability initiatives?
Overall, companies should commit to using AI thoughtfully and develop it to run in the most efficient ways possible:
- Develop the appropriate model for the use case – It’s important to tailor models and datasets to a use case’s goal and avoid going overboard training a system with irrelevant data and parameters. Providing the entirety of the internet for a generative AI model will consume huge amounts of computing power, which is not necessary for a generative AI intended only for responding to queries on data from a single company. Similarly, a predictive AI model will do just fine optimizing a data center’s electricity usage without training on data tracking seismic activity in the Earth’s crust.
- Evaluate the parameters required – Assess the trade-offs between accuracy and efficiency when deciding how many parameters to include. Do you really need to use 175 billion parameters, or can you achieve nearly the same level of accuracy with fewer parameters?
- Re-train models carefully – Decide on the frequency of re-training that is necessary, and schedule training in regions that are powered by renewable energy and/or at off-peak times, when regions are less likely to have to pull from fossil fuels to handle variable demand. Think critically about when, where, and how often you re-train your models.
- Run inferencing in regions with clean energy – Inferencing requires more processing power than traditional queries when processing data, so companies should consider directing inferencing traffic to locations running on clean energy. Although individual inferences are less compute-intensive than training, they can become larger cumulatively as the generative AI model continues to run.
- Don’t fall into the AI hype – Understand your problem, and design to the specific use case in mind. Avoid continuous data ingestion for simpler problems that may not require it.
Generative AI is already revolutionizing how humans do work, but we must be mindful of its impact on the climate as we make strides towards a more sustainable future.
[1]https://www.sciencedaily.com/releases/2021/09/210910121715.htm
[2] Updated report from Sept 2020 (see Abridged Report pgs 17-18: https://www.src.org/about/decadal-plan/
[3]https://arxiv.org/ftp/arxiv/papers/2104/2104.10350.pdf
[4]https://www.eia.gov/tools/faqs/faq.php?id=97&t=3#:~:text=In%202021%2C%20the%20average%20annual,about%20886%20kWh%20per%20month
[5]https://arxiv.org/pdf/2205.01068.pdf
[6]https://research.facebook.com/publications/sustainable-ai-environmental-implications-challenges-and-opportunities/
[7]https://openai.com/research/ai-and-compute
[8]https://semiengineering.com/ai-power-consumption-exploding/
[9]https://aws.amazon.com/machine-learning/elastic-inference/
[10]https://www.forbes.com/sites/moorinsights/2019/05/09/google-cloud-doubles-down-on-nvidia-gpus-for-inference/?sh=56c3e48f6792
[11]https://www.techrxiv.org/articles/preprint/The_Carbon_Footprint_of_Machine_Learning_Training_Will_Plateau_Then_Shrink/19139645;
[12]https://research.facebook.com/publications/sustainable-ai-environmental-implications-challenges-and-opportunities/
[13]https://themarkup.org/hello-world/2023/04/15/the-secret-water-footprint-of-ai-technology
The views and opinions expressed herein are the views and opinions of the author and do not necessarily reflect those of Nasdaq, Inc.