GreenGrid-Arm: Energy-Aware Cloud AI & Inference Profiler

An audit-ready profiler bridging Arm Neoverse Cloud LLM inference with dynamic grid energy telemetry via Arm Performix to measure vector pipe saturation, energy/token, and ISO/IEEE quality scores.

Comment

Inspiration

As global cloud infrastructure rapidly transitions to Large Language Model (LLM) inference workloads, the datacenter power and carbon bottleneck has become acute. Standard cloud monitoring tools track macro-level CPU utilization, but fail to correlate instruction-level hardware performance—such as Neon/SVE vector pipe saturation, memory bandwidth limits, and IPC—with time-varying power grid carbon intensities and energy consumption.

GreenGrid-Arm was built to solve this challenge on 64-bit Arm cloud infrastructure (AWS Graviton3/4, Ampere Altra, Arm Neoverse cores) by bridging physical power grid carbon dynamics with digital silicon telemetry.

What it does

GreenGrid-Arm is an audit-grade spatiotemporal profiler and interactive 3-screen wizard:

  1. Instruction-Level Hardware Profiling: Captures low-level hardware performance counters (Instructions Per Cycle [IPC], Neon/SVE SIMD vector pipe saturation) across execution targets (FP32 Baseline, FP16 Neon Vectorized, and INT4 SVE Peak Vectorized).
  2. Token-Level Energy Metrology: Tracks active power draw in Watts during inference to derive exact token-level energy consumption in Joules/token (( \text{J/tok} )) and Time-to-First-Token (TTFT) latency.
  3. Power Grid Carbon Integration: Integrates regional power grid carbon intensity (( \text{gCO}_2/\text{kWh} )) to evaluate the environmental footprint of cloud compute.
  4. Energy & Compute Quality Index (ECQI): Synthesizes microarchitecture saturation, throughput, and carbon intensity into a unified 0–100 score.
  5. Differential Audit Analysis & Export: Features a comparative delta viewer and exports standardized audit payloads in IEEE 3800 JSON and ASCII text formats.

How we built it

  • Application Framework: Built in Python using Streamlit with a clean, 3-screen guided wizard layout backed by st.session_state.
  • Hardware & Energy Modeling: Modeled Arm Neoverse V2 pipeline dynamics, Neon 128-bit / SVE2 256-bit vectorization throughput, and token energy intensity.
  • Interactive Data Visualization: Custom Plotly bar charts comparing inference throughput vs. vector pipe saturation and energy draw per token.
  • Audit & Session Persistence: Implemented a session-state history buffer to execute head-to-head run comparisons and generate real-time deltas (( \Delta \text{Tokens} ), ( \Delta \text{Latency} ), ( \Delta \text{Joules} ), ( \Delta \text{ECQI} )).

Challenges we ran into

  • Session State Synchronization: Preventing state resets during Streamlit re-runs when users clicked download buttons required careful session state persistence logic (st.session_state.has_run_profiler).
  • Standards Mapping: Mapping proprietary composite metrics back to formal ISO/IEC and IEEE data center standards required careful mathematical formulation to ensure compatibility with ISO/IEC 30134 and ISO/IEC 21836 frameworks.

Accomplishments that we're proud of

  • Standards Alignment: Successfully aligned the profiler with ISO/IEC 30134 (Data centre KPIs), ISO/IEC 21836 (Server active energy effectiveness), and IEEE 3800 (Data trading architecture).
  • Machine-Readable Telemetry: Engineered one-click export buttons producing validated IEEE 3800 JSON payloads for green compute credit trading and ASCII findings reports for human auditors.
  • Comparative Run Analysis: Built a dynamic differential analysis table that calculates real-time hardware performance deltas across multiple execution passes.

What we learned

  • Deepened insight into Arm Neoverse scalable vector extensions (SVE2) and SIMD pipeline saturation dynamics.
  • Learned how microarchitecture-level optimizations (moving from FP32 to INT4 SVE) directly translate into massive carbon displacement on high-emission power grids.

What's next for GreenGrid-Arm: Energy-Aware Cloud AI & Inference Profiler

  • Hardware Integration: Connect directly with live hardware counters on AWS Graviton4 instances using Linux perf and Arm Performix profiling hooks.
  • Positive Energy District (PED) Coupling: Expand the framework into urban microgrids to enable dynamic demand-side response and data center thermal reuse scheduling.

Built With

  • arm-performix
  • arm64
  • bash
  • c++
  • cloud-ai
  • energy-efficiency
  • ieee-standards
  • llama-cpp
  • neon
  • neoverse
  • python
  • simd
  • sve
Share this project:

Updates

Submission history