Inspiration
Modern engineering teams use a fragmented mix of cloud APIs, terminal agents like Claude Code, and local models via Ollama leaving FinOps and engineering leads completely blind to real-time spend until an unexpected invoice arrives. Existing observability tools act like surveillance proxies by logging sensitive prompts and proprietary source code to third-party clouds, which creates a massive security barrier. We wanted to build telemetry.ai: a local-first, zero-prompt AI FinOps command center that gives teams complete visibility into every token spent without ever touching user data.
What it does
telemetry.ai unifies cloud API usage, developer CLI telemetry, and local model inference into a single real-time command center:
Unified Multi-Provider Tracking: Normalizes token counts, caching efficiency, and 2026 model pricing across Google Gemini, Anthropic Claude Code, and local Ollama models (calculating avoided cloud spend for local inference).
Privacy-First Metadata Boundary: Captures strictly integer token counters, model identifiers, and timestamps locally storing zero prompt text, responses, or source code. Shadow AI Attribution: Flags unattributed API keys and developer machines so teams can map mystery usage to specific teams and workloads.
Autonomous Gemini FinOps Copilot: Uses the Google Gemini API to analyze grouped usage rollups, surface spend anomalies, auto-infer Shadow AI ownership, and generate interactive cost-saving playbooks (such as model right-sizing and prefix caching).
Budgets, Alerts & Exports: Tracks monthly team budgets with Slack Block Kit alert previews and 1-click normalized CSV exports.
How we built it
Backend & Ingestion Engine: Built with Python 3.14 and FastAPI, backed by a local SQLite (telemetry.db) persistence layer. We implemented a native OpenTelemetry (OTLP HTTP/JSON) receiver for Claude Code, a transparent SDK pass-through proxy and gateway for Google Gemini (google-genai), and an automated daemon watcher polling Ollama's local runtime (localhost:11434/api/ps).
AI Optimization Brain: Wired Google Gemini (gemini-3.1-flash-lite / gemini-3.8-flash) via structured outputs to evaluate aggregated SQL telemetry and generate actionable FinOps recommendations.
Frontend Dashboard: Built with React 19, TypeScript, Vite, Tailwind CSS, and Recharts featuring 3-second live auto-sync polling, stacked provider trendlines, interactive savings simulators, and inline attribution controls.
Challenges we ran into
Preventing Double-Counting While Typing: Tools like Claude Code broadcast running session token totals every few seconds even while a developer is still typing and hasn't hit Enter yet. Simply logging every update caused massive duplicate counting. We solved this by building a state tracker in our backend that remembers the previous session total and only records the net difference once a prompt actually finishes and generates a response.
Automatic Tracking Without False Alarms: Early on, watching local app files to detect AI usage caused false alarms whenever a code editor auto-saved a draft while typing. We solved this by switching to official telemetry streams, lightweight local routing, and direct status checks on local models capturing real AI usage automatically with zero false triggers.
Keeping the Dashboard Fast During API Spikes: Standard AI libraries automatically pause and retry in the background if Google's servers are momentarily busy, which froze our dashboard updates. We re-engineered those calls to run asynchronously with strict time limits so the interface stays fast and responsive even during traffic spikes.
Accomplishments that we're proud of
True Zero-Prompt Privacy Guarantee: Proving that enterprise-grade FinOps attribution, anomaly detection, and model optimization can be achieved using strictly statistical token shapes without storing a single byte of user prompts or code.
100% Live, Real-Time Multi-Surface Capture: Watching real token deltas stream into the dashboard within seconds from live Google Gemini calls, terminal Claude Code sessions, and local Ollama runs side-by-side.
Actionable FinOps Copilot: Going beyond passive charts by using Gemini to quantify exact monthly dollar savings from model routing and context caching.
What we learned
Running Totals vs. New Changes: Telemetry tools often send your entire session total over and over rather than just the tokens from your latest prompt. If you don't subtract the previous total from the new one, you'll accidentally double-count usage and get the wrong bill.
Showing the Dollar Value of Local AI: Running free, open-source models on your own laptop with Ollama usually goes unnoticed by finance teams. Calculating what those exact same prompts would have cost on a paid cloud API turns local AI into a clear, measurable cost-saving win.
Privacy Gets Teams on Board: Developers and security teams hate tools that spy on their work. By keeping all data local and strictly counting token numbers never saving actual prompts or source code teams feel safe turning tracking on.
What's next for telemetry.ai
Private Team-Wide Rollups: Letting developer laptops securely send anonymous, team-tagged token totals to a shared company dashboard giving managers a big-picture view of spend while keeping detailed personal logs locked on each developer's machine.
Smart Budget Auto-Routing: Turning our local Gemini gateway into an automatic traffic controller that switches simple tasks (like quick summaries) to cheaper, faster models whenever a team gets close to its monthly budget limit.
Tracking Automated GitHub & CI/CD Bots: Monitoring the AI tokens burned behind the scenes by automated pull-request reviewers, test writers, and overnight code checks.
Auto-Discovering Active Tools Only: Keeping the dashboard clean by automatically showing provider badges and charts only for AI tools you actually use instead of cluttering the screen with empty placeholders for unconnected services.
Built With
- anthropicapi
- claudecode
- fastapi
- geminiapi
- google-genai
- lucidereact
- ollama
- opentelemetry(otlp)
- python
- react
- recharts
- rest
- sql
- sqlite
- tailwind
- typescript
- uvicorn
- vite6