AquaRoute

Landing Page1
Landing Page2
Feature1: Routing Engine
Feature1: Prompting
Routing Engine: Small Model
Side by Side Comparison of small vs large model
Generated telemetry receipt
Feature2: Trace Stream
Feature3: Water Impact Matrix
Feature3: Settings

AI is becoming increasingly capable, but that capability comes with a computational cost that is often invisible to the developer. A simple factual question and a complex reasoning task can be treated the same way: send both to the largest model available and let it run.

We wanted to explore a different approach. Instead of asking only “Which model gives the best answer?”, we asked, “How much computation does this request actually need?” That question led us to AquaRoute, a resource-aware AI infrastructure layer that makes model selection, semantic caching, and estimated resource impact visible at the request level.

Our inspiration came from the idea that sustainability should not have to be a separate reporting layer added after an AI system is built. It can become part of the infrastructure decision itself.

AquaRoute sits between an AI application and the model provider as an intelligent gateway. For every request, AquaRoute:

->Classifies the request based on its complexity. ->Checks a semantic cache for a previous response to a similar request. ->Routes the request to either a smaller or larger model tier. ->Tracks actual token usage returned by the model. ->Estimates the request's energy and water impact using configured per-token resource assumptions. ->Explains the routing decision so developers can understand why a particular model was selected. ->Visualizes the entire request lifecycle through a live infrastructure dashboard.

If a request can be served from the semantic cache, AquaRoute avoids another model inference and records zero additional model-computation impact for that request in its estimator. The result is not just an AI response it is a transparent request-level telemetry trail showing what happened, why it happened, and what resource impact was estimated.

We built AquaRoute as a modular AI gateway using Node.js and Express, with Gemini as the model provider.

The core pipeline is:

Request ↓ Classifier ↓ Semantic Cache ↓ Router ↙ ↘ Small Large Model Model ↘ ↙ Resource Estimator ↓ Response + Telemetry

The classifier currently uses heuristic complexity signals such as prompt length and keywords associated with reasoning, architecture, analysis, coding, and other complex tasks.

For semantic caching, we generate embeddings using Gemini's embedding model and compare the incoming request against previously cached requests using cosine similarity.

Once a model responds, AquaRoute uses the actual token count returned by Gemini together with our configured per-token resource-intensity assumptions to calculate the estimated resource impact. We also built a live dashboard around the backend pipeline. Instead of presenting sustainability as a static number, the interface shows the current routing decision, cache status, model used, token usage, resource estimate, and request trace as the request is processed.

One of our biggest challenges was deciding how to estimate environmental impact without pretending that we could directly measure what happens inside a datacenter. Water and energy consumption depend on many factors outside our application layer, so we deliberately treated our numbers as estimates based on configured resource-intensity assumptions, rather than claiming they were direct physical measurements.

Another challenge was making the routing system useful without turning it into an unnecessarily complicated machine-learning system. For the prototype, we chose a transparent heuristic classifier so that every routing decision could be explained and demonstrated clearly.

We also had to think carefully about semantic caching. Exact string matching would miss requests that are phrased differently but ask essentially the same thing, so we introduced embedding-based similarity instead.

Finally, building a live infrastructure dashboard required us to make sure the visualizations represented actual pipeline events rather than simply being animations that looked convincing.

We're proud that AquaRoute evolved beyond a simple model-switching script into a working AI infrastructure gateway with observability.

Some of the key things we implemented are:

⚡ Dynamic small/large model routing 🧠 Semantic similarity-based caching 📊 Live request-level telemetry 💧 Estimated water and energy accounting 🔍 Explainable routing decisions 📡 Real-time processing states through Server-Sent Events 📈 AquaRoute vs. always-large comparison 🖥️ A developer-focused infrastructure dashboard

We also ran a five-prompt experiment comparing AquaRoute against an always-large baseline. Under our configured resource-intensity assumptions, AquaRoute produced 68.57 mL of estimated water impact versus 83.68 mL for the always-large baseline — an 18.1% reduction for that experiment.

Importantly, we treat this as an experimental result rather than a universal claim about AI resource consumption. We learned that building a sustainability-focused AI system is not just about calculating a number at the end.

We also learned that transparency matters. A developer is much more likely to trust a routing system when they can see why a request was routed to a particular model, whether it was served from cache, how many tokens were used, and how the estimate was calculated.

From a technical perspective, the project gave us practical experience with AI gateways, model routing, embeddings, semantic similarity, caching, Server-Sent Events, API integration, and request-level observability.

Most importantly, we learned to be careful about environmental claims. Resource estimation can provide useful engineering signals, but it should not be presented as a direct measurement of physical datacenter impact.

The next step is to move from a heuristic prototype toward a more robust resource-aware routing engine.

We want to explore: ->More sophisticated request-complexity classification ->Quality-aware routing that considers whether a smaller model can satisfy the request ->Persistent and distributed semantic caching ->More accurate and configurable resource-intensity models ->Larger benchmark datasets across different workloads ->Latency, cost, quality, and resource usage as combined routing signals ->Support for additional model providers ->Deeper request analytics and historical resource tracking ->Automated benchmarking of routing strategies

Ultimately, we envision AquaRoute as an infrastructure layer where resource efficiency becomes another first-class consideration alongside latency, cost, reliability, and response quality.

The goal isn't to tell developers which model they must use. It's to give them the infrastructure and visibility to make better-informed computation decisions for every request.