Inspiration
Every engineering team we spoke to had the same problem: they'd receive a $4,000 monthly AI API invoice with a single line item and no breakdown. Which feature caused it? Which model? Which user? Nobody knew. Teams were flying blind on one of their fastest-growing infrastructure costs.
We were inspired by how observability tools transformed cloud infrastructure spending in the 2010s — DataDog for servers, Segment for events. AI API costs needed the same treatment. LLM Ledger is that tool.
What it does
LLM Ledger is a real-time cost intelligence dashboard for engineering teams using multiple AI APIs simultaneously.
It tracks every AI API call — model used, feature that triggered it, tokens consumed, cost incurred, latency, and success status — and surfaces this data as actionable insights:
- Dashboard: Total spend, API call volume, active models, and cost-per-token trends over 30 days
- Models page: Head-to-head cost and performance comparison across GPT-4, Claude, Gemini, Llama, and Mistral
- Analytics page: Monthly spend trends correlated with call volume
- Alerts page: Budget thresholds per feature or model with real-time burn rate indicators
- Settings page: 3-line SDK integration snippet that any engineering team can drop into their codebase
The result: engineering teams stop getting surprised by their AI bill.
How we built it
Frontend: Built with Next.js and scaffolded using Vercel v0, deployed on Vercel. UI components via shadcn/ui and Tailwind CSS. Charts built with Recharts.
Backend: Two Vercel API routes handle all data flow:
- /api/log — receives POST requests from the SDK and writes structured event records to DynamoDB
- /api/analytics — scans DynamoDB and returns aggregated cost breakdowns by model, feature, and user
Database: Amazon DynamoDB in Asia Pacific (Mumbai), on-demand capacity mode. The table schema uses event_id as partition key and timestamp as sort key. Three Global Secondary Indexes enable instant aggregation:
- GSI 1: PK = team_id (query all events for a team)
- GSI 2: PK = feature_tag (query cost by feature)
- GSI 3: PK = model (query cost by model)
DynamoDB was a deliberate architectural choice — not a default. High-frequency telemetry data (one record per AI API call, potentially thousands per minute) requires a database that handles write throughput without provisioning headaches. DynamoDB's on-demand mode and GSI design gave us exactly that.
Challenges we ran into
The hardest challenge was the DynamoDB data modeling. Our first instinct was to use a relational schema with aggregation queries — but DynamoDB is not a relational database. We had to redesign around access patterns first, then build the schema to serve those patterns. The GSI design went through three iterations before we landed on one that supported all our query needs without table scans.
Parsing DynamoDB's native Number type in the API response was a subtle bug that cost us time — DynamoDB returns numbers as strings in the N attribute, requiring explicit parseFloat() conversion that isn't obvious from the documentation.
Building a full-stack B2B SaaS in a single hackathon sprint also required ruthless scoping. We cut user authentication, webhook alerting, and multi-tenant isolation to ship a clean, focused demo.
Accomplishments that we're proud of
The architecture diagram tells the story we're most proud of: every technology choice in LLM Ledger was deliberate.
DynamoDB was chosen because it is the correct database for event telemetry at scale — not because it was convenient. The GSI design means we never do a full table scan for any production query. The Vercel deployment means zero infrastructure management for the teams using this.
We're also proud of the UI quality. The dashboard looks like a product you'd pay for — dark theme, real chart data, budget alert progress bars, SDK integration snippet. It doesn't look like a hackathon project.
Most importantly: the problem is real. Every person on an engineering team using AI APIs right now has this pain. LLM Ledger is a product that should exist.
What we learned
Three things stood out:
DynamoDB's access-pattern-first design philosophy is genuinely different from relational thinking. You can't bolt on query flexibility after the fact — you design for your queries upfront, then build the schema. This was a mental model shift that made the final architecture much cleaner.
Vercel's v0 dramatically accelerates frontend development but requires careful integration work on the backend. The frontend scaffolding was production-quality within minutes; the API layer required precise attention to how Next.js API routes interact with environment variables in the Vercel runtime.
The hardest part of building a developer tool is making the value proposition obvious in 30 seconds. We went through five versions of the dashboard layout before landing on one where a first-time visitor immediately understands what they're looking at.
What's next for LLM Ledger — AI API Cost Intelligence
The roadmap is clear:
Near term: Real SDK package published to npm so teams can instrument in one line. Webhook alerting when spend thresholds are breached. Multi-tenant authentication with team isolation in DynamoDB.
Medium term: Anomaly detection — flag unusual spend spikes automatically using statistical baselines stored in DynamoDB. Cost forecasting based on 30/60/90-day trends.
Long term: Direct API integrations with OpenAI, Anthropic, and Google billing APIs to automatically reconcile logged spend against actual invoices. Enterprise SSO and audit logs.
The core insight driving all of it: AI infrastructure costs are the new cloud costs. Every company that ran into surprise AWS bills in 2015 is going to run into surprise AI bills in 2025. LLM Ledger is the tool that prevents that.
Log in or sign up for Devpost to join the conversation.