Inspiration
AI data centers are on track to use as much electricity as Japan by 2030 (about 945 TWh, per the IEA), and they draw it hardest at peak hours, when the grid is most strained. Yet every AI request runs immediately, on the biggest model, at full precision, whatever the grid is doing. We wanted AI to act as a flexible load that gives power back when the grid needs it most.
What it does
GridAware sits in front of a self-hosted AI model and reads New York's power grid live every five minutes.
- Grid relaxed: runs the full model.
- Grid stressed: switches to smaller or quantized models, sends only hard questions to the big model, and holds deferrable jobs until the grid calms down.
Every answer shows the energy it used, the model tier that handled it, and its savings compared with running everything at full power.
How we built it
- Models: Qwen2.5 served with vLLM on a Vultr GPU, in FP16, INT8, INT4 and a small 1.5B tier.
- Sense: live NYISO fuel mix, prices and load, plus the GPU's real power draw measured with NVML.
- Understand: a LightGBM forecaster predicts grid stress, and a fast classifier, distilled from Gemini labels, predicts which tier can answer each question correctly.
- Explain: Gemini explains each decision in plain English, and ElevenLabs reads out spoken grid updates.
Energy per request is measured directly from GPU power:
$$E_q = \int_{t_0}^{t_1} P_{\text{GPU}}(t)\,dt$$
Carbon uses the grid's live average intensity, computed from the fuel mix:
$$I(t) = \frac{\sum_f G_f(t)\,e_f}{\sum_f G_f(t)}, \qquad C_q = E_q \cdot I(t)$$
where $G_f$ is generation from fuel $f$ and $e_f$ is its emission factor. The router picks the lowest-energy tier that stays within the quality budget $\epsilon$:
$$k^* = \arg\min_k E_k \quad \text{s.t.} \quad \text{Acc}_{\text{base}} - \text{Acc}_k \le \epsilon$$
Challenges we ran into
- Attributing GPU energy to individual requests when several run in the same batch.
- Fitting several model tiers into GPU memory at once.
- Matching grid data to where the compute physically runs. Vultr's GPUs are not on New York's grid, so the New York data center scenario is simulated.
- Keeping the router fast enough that it doesn't eat into the savings.
What we learned
- The carbon from a single AI query is tiny. The real impact is cutting power at peak and saving compute at scale.
- Quantization and difficulty routing save a lot of energy with little quality loss on easy queries.
- Average and marginal emissions differ, and honest accounting matters.
What's next for GridAware
- Marginal-emissions data instead of grid averages.
- Integration with official grid demand-response programs.
- Routing work across data centers on different grids (NYISO, PJM, IESO).
- A plugin that any team running vLLM or Kubernetes can drop into their stack.
