Finference

Finference connects AI costs to revenue, finds margin leaks, governs safer model routing, and turns usage into billable units—raising margin from 51.8% to 61.8%.

Comment

Inspiration

AI SaaS companies see revenue immediately, but inference costs arrive later as blended provider invoices. Teams often cannot tell which customer, feature, or model route is profitable until margins have already deteriorated.

We built Finference to make AI unit economics operational—not another month-end spreadsheet.

What it does

Finference connects every AI request to:

  • Customer revenue
  • Inference cost
  • Token usage
  • Model and provider
  • Feature and customer
  • Latency and reliability

It identifies margin leaks and recommends more efficient model routes. Before activation, each recommendation is tested against historical traffic, quality thresholds, latency limits, and rollback conditions. An authorized operator must approve the change.

The same authoritative ledger also converts usage into customer-facing billing units.

In our demonstration, Finference reduces monthly inference cost from $13,751 to $10,887, increasing gross margin from 51.8% to 61.8% and protecting $2,864 per month.

How we built it

Finference uses:

  • Next.js 16, React 19, TypeScript, Tailwind CSS, and Recharts
  • Neon Postgres for durable multi-tenant state
  • Neon Auth for managed authentication and protected workspaces
  • Drizzle ORM with versioned database migrations
  • Zod for strict API validation
  • SHA-256 and HMAC-SHA256 for API-key storage and signed ingestion
  • Stripe Checkout and Billing Meter adapters for subscription and usage billing
  • Backboard assistant and thread adapters for persistent agent memory
  • Vitest, ESLint, GitHub Actions, and Vercel for verification and deployment

The live application persists workspaces, policies, economic events, API-key hashes, audit evidence, rate limits, webhook records, and meter batches.

Challenges we faced

The hardest challenge was preventing AI optimization from becoming an unsafe black box. Choosing a cheaper model is easy; proving that it preserves quality, restricting its traffic exposure, enforcing authorization, recording the evidence, and defining automatic rollback conditions required a complete governance loop.

We also had to make ingestion, billing, and webhook processing safe under retries and concurrent requests. Database uniqueness constraints, deterministic meter watermarks, atomic policy transitions, and replay-safe webhook handling became essential.

Finally, we chose honest degradation over simulated integrations. Neon Postgres and authentication are live. Stripe and Backboard are fully implemented adapters, but the deployment clearly reports them as adapter-ready until external account credentials are installed.

Accomplishments

  • Built a real authenticated product rather than a static concept page
  • Created a durable, tenant-scoped economic event ledger
  • Implemented one-time API keys stored only as SHA-256 hashes
  • Added persistent serverless rate limiting and idempotent ingestion
  • Built human-governed routing policies with simulation evidence
  • Connected policy decisions to an immutable audit trail
  • Implemented durable usage aggregation and Stripe-ready billing
  • Persisted all 100 events in a production concurrency test with zero failures
  • Achieved over 95% coverage on core financial and security logic
  • Reached zero known dependency vulnerabilities

What we learned

AI FinOps becomes substantially more valuable when cost is connected to revenue. The correct optimization target is not the cheapest token—it is contribution margin under explicit quality constraints.

We also learned that production credibility comes from details that are easy to overlook: authentication, tenant isolation, idempotency, permissions, durable state, webhook replay protection, and truthful capability reporting.

What's next

  • OpenTelemetry and provider SDK instrumentation
  • Online quality evaluation and canary routing
  • Runtime policy distribution and automatic rollback execution
  • Stripe reconciliation and invoice-preview interfaces
  • SSO, SCIM, retention controls, and regional data residency
  • Kafka-based ingestion and ClickHouse analytics at larger scale

Finference turns delayed AI cost into a governed, revenue-aware operating loop: observe, optimize, approve, and bill.

Built With

  • backboard
  • docker
  • drizzleorm
  • githubactions
  • neonauth
  • neonpostgres
  • next.js
  • playwright
  • react
  • recharts
  • stripebilling
  • tailwind
  • typescript
  • vercel
  • vitest
  • zod
Share this project:

Updates

Submission history