Microsoft Puts AI Query Power Use at 0.16 to 0.60 Wh, Far Below Prior Estimates
The revised figures challenge widely cited claims that a single AI query uses several watt-hours, with implications for data center planning and public perception of AI energy costs.
Key Facts
- Microsoft estimates a typical AI query uses 0.16 to 0.60 Wh of power, based on a model with over 200 billion parameters running on eight NVIDIA H100 GPUs.
- The median power consumption for a 300-token query is 0.31 Wh, with water cooling estimated at 0 to 0.067 mL per query.
- Microsoft says earlier estimates overstated AI query power use by 4 to 20 times because they ignored batch processing and efficient GPU operation.
- Queries requiring multi-step reasoning or long code generation, with a median response of 5000 tokens, consume about 3.91 Wh, roughly 13 times a typical query.
Reporting from 1 source: GIGAZINE.
Microsoft published an analysis estimating that a typical query to a large language model consumes 0.16 to 0.60 Wh of power, roughly equivalent to running a microwave for a few seconds. The figure is 1/20th to 1/4th of earlier estimates, which Microsoft says failed to account for batch processing and production GPU utilization rates.
Microsoft's research team based the estimate on a model with over 200 billion parameters running on a server with eight NVIDIA H100 GPUs. They combined metrics including tokens processed per second, per-server power consumption, and facility-level Power Usage Effectiveness (PUE). For a median query of about 300 tokens, median power consumption was 0.31 Wh, with the central 50% of the range falling between 0.16 and 0.60 Wh. Water consumption for cooling was estimated at 0 to 0.067 mL per query.
Microsoft argues that earlier estimates-some suggesting a ChatGPT query uses roughly 10 times the power of a Google search-overstated consumption by 4 to 20 times because they did not reflect batch processing or efficient GPU operation in high-user environments. However, the company notes that queries requiring multi-step reasoning or long code generation, with a median response of 5000 tokens, consume about 3.91 Wh, roughly 13 times a typical query.
Synthesized by Yomimono from the 1 cited source below, including Japanese-language reporting where cited, then editorially reviewed before publishing.