routewise

Inspiration

Modern AI applications often use powerful, expensive language models for every task, even when a smaller and cheaper model could handle the request successfully. This leads to unnecessary API costs and resource consumption.

We wanted to solve this problem by building an intelligent routing system that selects the right AI model based on the complexity of a task.

That idea inspired us to create RouteWise — Make Every Model Earn Its Cost.

Our goal was to demonstrate that AI applications can become more cost-efficient without blindly compromising on output quality.

What it does

RouteWise is a cost-aware, verification-driven LLM routing system designed for invoice data extraction.

It follows a three-tier model architecture:

Tier 1 – Small Model: Starts with a low-cost model for invoice extraction.
Tier 2 – Medium Model: Escalates when the first model fails verification.
Tier 3 – NVIDIA Model: Uses NVIDIA Nemotron as the final escalation tier for more challenging cases.

Before accepting an extracted invoice, RouteWise performs three important verification checks:

Schema Validation: Checks whether the extracted information follows the required invoice structure.
Arithmetic Validation: Checks whether line-item amounts, subtotal, tax, and total are mathematically consistent.
Source Grounding: Checks whether extracted values can be found in the original invoice text.

If verification succeeds, the result is returned. Otherwise, the system escalates to the next model tier.

The dashboard also provides a routing trace with model selection, token usage, latency, estimated API cost, and verification decisions.

How we built it

We developed RouteWise using a combination of Python-based backend technologies and a modern React frontend.

Technology Stack:

Python for the core routing and evaluation logic.
FastAPI for backend APIs.
React and Vite for the interactive dashboard.
Nebius Token Factory for accessing hosted language models.
NVIDIA Nemotron as the final model in our configured model ladder.
Pydantic for structured invoice schema validation.
HTTPX for asynchronous API communication.
PyYAML for configuration-driven model selection and pricing.

We generated a reproducible synthetic dataset containing 150 invoices, divided into 50 training records and 100 testing records.

We also developed an evaluation framework to compare four strategies:

Always-small model
Always-medium model
Always-large model
RouteWise adaptive routing

The evaluation framework is designed to measure field accuracy, exact-match rate, API cost, and latency.

The application includes a FastAPI backend, a React dashboard, automated tests, GitHub Actions CI, and Docker-based deployment configuration.

Challenges we ran into

One of our biggest challenges was ensuring that a model's output was actually correct rather than simply valid JSON.

A model can produce correctly structured invoice data while still extracting incorrect amounts or information that does not exist in the original document.

We addressed this by implementing multiple verification layers for schema validity, arithmetic consistency, and source grounding.

Another challenge was designing an escalation mechanism that balances accuracy, latency, and API cost.

We also worked on making model selection and pricing configurable instead of hardcoding them into the application.

Finally, access to live API credentials was a limitation during development, so completing real-world benchmarking remained a challenge.

Accomplishments that we're proud of

Built an end-to-end LLM routing application with a three-tier model architecture.
Integrated a configuration-driven model strategy using Nebius Token Factory.
Included NVIDIA Nemotron in the final escalation tier.
Implemented deterministic invoice verification.
Generated a reproducible dataset of 150 invoices.
Developed a cost and latency tracking mechanism.
Created an interactive dashboard with a routing trace.
Implemented a benchmark framework for comparing four model strategies.
Successfully passed our initial automated tests, lint checks, and frontend production build.

We are especially proud of building a system that makes AI model selection transparent and treats verification as an essential part of the AI workflow.

What we learned

Through this project, we learned that using a larger AI model does not always mean using a better solution.

We understood the importance of structured outputs, deterministic verification, model orchestration, token-based cost estimation, and asynchronous API communication.

We also learned that building a reliable AI application requires more than connecting an API. It requires a complete system to evaluate, validate, monitor, and improve model decisions.

What's next for RouteWise

Our next goal is to complete live benchmarking using the Nebius API and validate RouteWise against real, representative invoice datasets.

We plan to:

Measure actual accuracy, latency, and API costs across all four strategies.
Improve the verification and escalation mechanisms.
Explore document upload and OCR-based invoice extraction.
Add human review for uncertain or high-risk extractions.
Improve the dashboard with real-time benchmark visualizations.
Explore additional NVIDIA models and more complex document-processing use cases.

Our vision is simple: Use the smallest capable model, verify every result, and spend more compute only when the task genuinely requires it.