Tollgate - Real-Time Margin Observability for AI Agents

Stop flying blind on AI profit. Tollgate joins LLM provider costs with your revenue plan to show live per-customer gross margin, so you catch Customer B losing you money before the invoice lands.

Comment

Inspiration

AI products have reached a turning point. More companies now price their offerings based on usage and business outcomes, whether that is per resolved ticket, per API call, or per agent execution. However, the true cost of delivering those services often remains hidden until the monthly LLM provider invoice arrives.

Reasoning tokens, tool invocations, retries, sub-agent execution, cache operations, and multimodal requests all contribute to infrastructure costs that are almost impossible to track manually in real time.

This led us to ask a simple question:

Why does every AI company know exactly what it charges customers, but very few know what each customer actually costs them?

I discovered a clear gap in the market.

Platforms such as Stripe, Orb, and Metronome focus on billing and revenue generation. LLM observability tools such as Helicone and Langfuse provide detailed cost tracing. What is missing is a platform that connects both sides to deliver live, customer-level gross margin visibility.

That blind spot in AI FinOps became the foundation for Tollgate.

The market is moving quickly. Y Combinator has backed startups focused on AI-native billing, while industry research shows that companies adopting hybrid pricing models achieve approximately 38% higher Net Revenue Retention (NRR). As usage-based pricing becomes the norm, understanding profitability in real time becomes essential.

What it does

Tollgate is a real-time gross margin observability platform built for AI agent companies.

Rather than replacing your billing system, Tollgate integrates with your existing stack and continuously calculates whether the prices you charge actually cover your LLM infrastructure costs. It provides profitability insights at the customer, agent, and execution level.

The Customer Support Agent Demo

Consider two customers on the exact same pricing plan of $0.50 per resolved support ticket.

Customer Workload Cost per Ticket Gross Margin
Customer A FAQ chatbot with lightweight requests ~$0.02 🟢 +96%
Customer B Billing disputes requiring heavy reasoning and tool usage ~$0.53 🔴 −6%

At first glance, the business appears healthy because the blended margin is roughly 40%.

Tollgate exposes the underlying reality. Customer B is operating at a loss, with reasoning tokens accounting for nearly 72% of total cost. Although both customers pay the same amount, their economics are completely different.

Instead of discovering this weeks later when the provider invoice arrives, Tollgate surfaces the issue immediately.

Key Features

Overview Dashboard

  • Live gross margin percentage
  • Recognized revenue
  • Provider costs
  • Number of unprofitable customers
  • Resolution rate with period-over-period trends
  • Revenue versus cost visualization
  • Cost breakdown by model, provider, and AI agent
  • Detailed token cost analysis covering input, output, reasoning, cached tokens, cache writes, audio, image, video, tool calls, and web search

Customer Analytics

  • Searchable and sortable customer list
  • Individual customer profitability
  • Per-agent margin analysis
  • Token cost breakdown

Execution Logs

  • Complete audit trail of every AI agent execution
  • Token-level usage details
  • Revenue, cost, gross margin, execution outcome, and provider information

Simulator

  • Replay historical workloads against hypothetical pricing models
  • Compare different LLM providers before making production changes
  • Adjust pricing plans and immediately evaluate their financial impact
  • View side-by-side comparisons of revenue, cost, and gross margin
  • Analyze customer-level margin changes before deploying pricing or model updates

Intelligent Alerts

Create real-time alerts for:

  • Negative gross margin
  • Reasoning token spikes
  • Low margin thresholds
  • Cost per run spikes

Alerts can be delivered through webhook and email.

Integrations Hub

  • API key management
  • Pricing plan management across six pricing models
  • Customer plan assignment
  • Notification channel configuration
  • SDK quick-start snippets

Model Pricing Catalog

  • Automatically synchronizes provider pricing daily
  • Supports pricing across 1,500+ LLM models from multiple providers
  • Tracks input, output, reasoning, cache read, and cache write pricing
  • Server-side cost calculations using the latest provider rate cards
  • Flags unmatched models to prevent inaccurate cost attribution
  • Maintains up-to-date pricing for accurate financial reporting

Settings

  • Multi-tenant RBAC
  • Owner, Admin, Member, and Viewer roles
  • SSO and OIDC
  • Audit logs
  • Secure cascading account deletion

Data Export

  • Export execution logs, cost reports, and gross margin reports
  • Filter exports using custom date ranges
  • Download reports in CSV format for further analysis
  • Generate finance-ready datasets for reporting and reconciliation

SDKs

Official Open Sourced SDKs for:

  • TypeScript (@tollgateai/sdk)
  • Python (tollgateai)

Both support automatic instrumentation for Anthropic, OpenAI, AWS Bedrock, and Google Gemini.

Tollgate SDK GitHub Repository

How I built it

Tollgate is built around a purposefully designed dual-database architecture, which serves as the technical foundation of the platform.

Dual-Database Architecture

Amazon DynamoDB

DynamoDB powers the real-time ingestion layer.

It stores high-throughput usage events, provides idempotent writes using conditional expressions, maintains margin rollups for sub-millisecond reads, and leverages DynamoDB Streams with TTL support for downstream processing.

Duplicate events are automatically rejected through idempotency keys, ensuring the same event is never processed twice.

Amazon Aurora PostgreSQL

Aurora Serverless v2 serves as the financial system of record.

It manages:

  • Pricing plans
  • Customers
  • Agent runs
  • Alerts
  • Audit events
  • API keys

Aurora also performs pricing joins, analytical queries, financial reporting, and multi-period trend analysis through SQL.

Aggregation Pipeline

Every /api/track request is written to DynamoDB.

Immediately afterward, an inline aggregateEvent() process updates Aurora so the dashboard reflects new events instantly, even in Vercel-only deployments.

In parallel, DynamoDB Streams trigger a Lambda-based aggregation service with its own Dead Letter Queue for resilient asynchronous processing.

Since both execution paths invoke the same idempotent aggregation function, running them simultaneously never results in duplicate financial calculations.

Cost Engine

The pricing engine independently calculates costs across twelve usage dimensions:

  • Input tokens
  • Output tokens
  • Reasoning tokens
  • Cached token reads
  • Cache writes (5-minute TTL)
  • Cache writes (1-hour TTL)
  • Audio input
  • Audio output
  • Image input
  • Image output
  • Video input
  • Web search
  • Tool calls

Financial values are stored using NUMERIC(20,6) precision in Aurora, eliminating floating-point rounding errors.

Continuous Integration includes truth tests using canonical execution scenarios to guarantee pricing accuracy.

Revenue Engine

Tollgate supports six pricing models:

  • Per unit
  • Per resolution
  • Usage-based
  • Per seat
  • Monthly subscription
  • Hybrid pricing

Revenue is recognized only after successful outcomes.

The customerFinancials() service combines activity-based revenue with subscription revenue so every dashboard metric remains internally consistent.

Frontend

  • Next.js 16 (App Router)
  • React 19
  • TypeScript
  • Tailwind CSS v4
  • Recharts

The public website is available at /, while authenticated users access the application through /dashboard.

Security

Security was designed into the architecture from day one.

Key protections include:

  • AES-256-GCM encryption with AWS KMS
  • Scrypt password hashing
  • HMAC session cookies
  • SHA-256 API key indexing
  • Complete tenant isolation through account_id
  • Prompt and completion data are never stored
  • Distributed API rate limiting using Upstash Redis

Infrastructure

Infrastructure is provisioned using Terraform.

This includes:

  • DynamoDB
  • Aurora Serverless v2
  • Lambda functions

Database migrations are version-controlled, and every pull request automatically runs linting, type checking, testing, and production builds.

Challenges I ran into

1. Sub-cent Cost Precision

LLM requests frequently cost only fractions of a cent.

Our original implementation stored costs as integer cents, causing nearly every execution to appear free and falsely report margins approaching 100%.

I redesigned the financial model to store costs with six decimal places while keeping revenue as whole cents. This required coordinated changes across validation, aggregation, storage, APIs, and UI formatting.

2. Provider-Specific Reasoning Tokens

Each provider exposes token usage differently.

Anthropic includes reasoning within output tokens.

OpenAI exposes reasoning separately.

AWS Bedrock uses a completely different response format.

Creating a unified token model required carefully mapping each provider's native response without making assumptions about unavailable fields.

3. Revenue Recognition

Subscription pricing models such as flat monthly and per-seat billing do not generate revenue on every execution.

I introduced a dedicated subscription revenue service that amortizes recurring revenue across billing periods and combines it with execution revenue to produce accurate financial reporting.

4. Multi-Tenant Security

Every API request, DynamoDB partition key, SQL query, and aggregation step is scoped to the authenticated account_id.

Client-provided customer identifiers are never trusted without ownership validation, ensuring complete tenant isolation.

Accomplishments that I am proud of

  • Designed a dual-database architecture that clearly separates high-throughput operational workloads from finance-grade analytics.
  • Developed a twelve-component cost engine backed by continuous integration truth tests.
  • Released production-ready SDKs @tollgateai/sdk on npm and tollgateai on PyPI - zero dependencies, auto-instrumentation for Anthropic, OpenAI, Bedrock, and Gemini.
  • Enforced privacy by architecture through schema validation that rejects prompt and completion content before storage.
  • Achieved a fully passing test suite covering cost calculations, idempotency, and tenant isolation.
  • Successfully deployed a live production application backed by DynamoDB and Aurora.

What I learned

Building financial software for AI systems requires a different engineering mindset.

Accuracy Matters More Than Availability

Displaying an incorrect gross margin is more harmful than displaying no margin at all.

Every API follows a simple rule: fail loudly instead of fabricating data.

Dual Databases Complement Each Other

DynamoDB provides exceptional ingestion performance, deduplication, and event streaming.

Aurora makes analytical queries, financial reporting, and pricing joins dramatically simpler.

Together they provide both operational speed and financial correctness.

Fractional-Cent Accounting Is Essential

Traditional finance systems assume integer-cent precision.

LLM workloads do not.

Separating fractional-cost accounting from whole-cent revenue proved to be the correct long-term design.

Pricing Maintenance Is the Real Challenge

Keeping pricing synchronized across Anthropic, OpenAI, Gemini, Bedrock, and gateway providers is significantly more complex than building the aggregation pipeline itself.

Automatic LiteLLM synchronization combined with manual overrides became essential.

AI-Assisted Development Accelerates Delivery

Using v0 allowed us to generate a polished production dashboard rapidly, giving us more time to focus on backend architecture, financial correctness, and system reliability.

What's next for Tollgate

Immediate Priorities

Design Partner Onboarding

Bring on 3–5 agentic AI startups to validate Tollgate against real production workloads, pricing models, and customer economics.

Stripe & Orb Revenue Connector

The infrastructure is already built and feature-flagged.

The next step is enabling real invoice synchronization so declared plan revenue is replaced by actual recognized revenue.

DynamoDB Streams Lambda Deployment

Transition from inline aggregation to the fully decoupled asynchronous processing pipeline backed by DynamoDB Streams and a Dead Letter Queue.

This provides guaranteed-delivery retries without depending on the Vercel request lifecycle.

Production Hardening

Nightly Provider Reconciliation

Automatically compare Tollgate's computed provider costs against actual monthly provider invoices, targeting less than 3% variance as an automated quality gate.

Aurora Read Replica

Introduce an Aurora read replica to isolate heavy analytical workloads, including 30-day trend analysis and margin distribution reports from the transactional write path.

Automated Email Delivery

Complete the Resend integration so team invitations are delivered automatically instead of relying on manually generated invite links.

Product Expansion

Cost Guardrails

Automatically recommend lower-cost models or throttle execution when a customer's per-run cost exceeds a configurable threshold, preventing margins from becoming negative.

Margin Simulator

Promote the simulator from a power-user feature into a first-class workflow, allowing founders to model pricing changes or LLM provider swaps before deploying them.

Board-Ready COGS Export

Generate reconciled per-customer cost attribution reports that finance teams can directly import into P&L statements.

Additional Provider Support

Expand native integrations to include:

  • OpenRouter
  • Nebius
  • LiteLLM Proxy
  • Azure OpenAI

Long-Term Vision

Multi-Region Aurora DSQL

Support globally distributed AI companies with multi-region write capability while maintaining financial consistency.

Agent Topology Maps

Visualize multi-agent execution graphs so founders can identify exactly which sub-agent is driving infrastructure costs.

AI-Powered Pricing Recommendations

Analyze historical margin distributions to recommend pricing adjustments, model changes, or provider optimizations before profitability begins to decline.

Tryout: Tollgate

Built With

Share this project:

Updates