Curious about the electricity your AI interactions consume? Hugging Face engineer Julien Delavande was, prompting him to develop a tool to uncover the answer.
Every operation of AI models demands energy. These models run on GPUs and specialized hardware that require substantial power to handle extensive computational tasks. Estimating the exact power consumption of these models is challenging, but it’s anticipated that the growing reliance on AI will significantly increase electricity demands in the coming years.
The rising energy needs for AI have driven some firms to adopt less eco-friendly practices. Tools like Delavande’s are intended to highlight this issue and perhaps encourage AI users to consider the environmental impact.
“Minor energy savings can become significant when multiplied by millions of queries — choices in models or output length can have substantial environmental consequences,” Delavande and his colleagues stated.
Delavande’s tool integrates with Chat UI, an open-source interface for models such as Meta’s Llama 3.3 70B and Google’s Gemma 3. It calculates the energy usage of messages sent to and from the model in real time, displaying the consumption in Watt-hours or Joules. The tool even compares the energy usage of models to that of everyday household appliances, like microwaves and LED lights.
For instance, the tool estimates that asking Llama 3.3 70B to draft a standard email consumes about 0.1841 Watt-hours, which equates to running a microwave for 0.12 seconds or operating a toaster for 0.02 seconds.
It’s important to note that these figures are estimates. Delavande acknowledges that they may not be precisely accurate, but they serve as a crucial reminder that everything, including chatbots, comes with an energy cost.
“With initiatives like the AI energy score and broader research into AI’s energy footprint, we aim to promote transparency within the open-source community. Eventually, energy usage might become as commonly visible as nutrition labels on food!” Delavande and his team expressed.
(Source: TechCrunch)
