CarbonCompute

💡 Inspiration

As Generative AI and Large Language Models (LLMs) scale rapidly across enterprises, their hidden environmental and financial costs skyrocket. Most organizations track API latency and token counts, but suffer from complete blind spots regarding energy consumption (Joules/kWh), real-time carbon emissions based on regional grid mixes, and compliance mandates like the EU CSRD.

We were inspired to build CarbonCompute to bridge this gap—providing a transparent, serverless telemetry and auditing engine that turns raw AI workloads into actionable GreenOps and FinOps insights.

🚀 What it does

CarbonCompute is an end-to-end serverless telemetry pipeline designed to monitor, calculate, and audit the environmental and financial impact of LLM inferences in real time.

Key features include:

• Asynchronous Telemetry Ingestion: Captures workload details (GPU types, token counts, execution duration, and datacenter region) via a secure API Gateway.
• Advanced GreenOps Engine: Computes energy consumption (incorporating PUE factors), algorithmic efficiency metrics (Tokens/Watt and Joules/Token), and accurate carbon emissions ($gCO_2e$) using dynamic regional grid carbon intensity factors.
• FinOps Integration: Estimates real-time electricity costs and cost-per-million tokens.
• Audit-Ready Data Lake: Automatically structures and partitions enriched telemetry data into Google Cloud Storage (GCS) for future compliance reporting (CSRD ESRS-E1, GHG Protocol Scope 2 & 3).

🛠️ How we built it

CarbonCompute leverages a robust, event-driven Google Cloud architecture:

• API Gateway / Ingestion: Secure entry point for client agents transmitting telemetry payloads.
• Google Cloud Pub/Sub (carbon-telemetry-topic): Decouples ingestion from processing to ensure high scalability and zero data loss.
• Cloud Functions Gen 2 Worker (process-carbon-telemetry): Built with Python 3.12 and Functions Framework, triggered asynchronously via Eventarc to process payloads, execute mathematical models, and evaluate compliance metadata.
• Google Cloud Storage (GCS): Acts as the partitioned raw data lake (raw/{YYYY}/{MM}/{DD}/{client_id}/{telemetry_id}.json) equipped with custom searchable blob metadata.

🧗 Challenges we ran into

• Event-Driven Configuration: Fine-tuning the asynchronous bindings between Cloud Pub/Sub topics and Cloud Functions Gen 2 via Eventarc to ensure low latency and reliable message delivery.
• Dynamic Grid Factors: Mapping multi-regional data center factors accurately while accommodating custom user-defined grid intensities.

🏆 Accomplishments that we're proud of

• Fully Operational Cloud Pipeline: Successfully deployed and tested a live serverless architecture on Google Cloud (us-central1).
• Multi-Dimensional Metrics: Successfully merging environmental KPIs (Joules, Tokens/Watt, CSRD) with financial metrics (FinOps cost estimation).
• Audit-Ready Design: Building a scalable data lake storage hierarchy that makes enterprise sustainability reporting seamless.

🔮 What's next for CarbonCompute

• Integrating Google BigQuery on top of the GCS data lake for advanced SQL analytics and executive dashboarding.
• Developing a drop-in Python SDK/Client wrapper for automatic telemetry tracking inside popular LLM frameworks (like LangChain or OpenAI clients).

Built With

• api
• eventarc
• finops
• framework
• functions
• git
• google-cloud
• greenops
• json
• platform
• pub/sub
• python
• rest
• serverless
• storage