Inspiration
As AI applications become more common in production, developers often struggle to answer simple but important questions:
- Which models are driving our costs?
- Why did spending suddenly increase?
- Which endpoints are slow or failing?
- How can we optimize our AI infrastructure?
Most teams only discover these issues after receiving expensive API invoices or manually analyzing logs. We wanted to build a platform that gives developers complete visibility into their AI systems in real time.
TokenWatcher was created to make AI observability simple, helping developers monitor usage, understand costs, and make data-driven optimization decisions before small problems become expensive ones.
What it does
TokenWatcher is an AI observability platform for production LLM applications.
It enables developers to:
- Monitor token usage and API costs in real time
- Track latency, request failures, and model performance
- Analyze usage by workspace, model, and endpoint
- Forecast future AI spending
- Detect unusual usage patterns
- Generate AI-powered optimization recommendations
- Receive reports and interact with their telemetry through Telegram
The platform also includes a lightweight SDK that allows developers to instrument their applications with minimal code changes.
How we built it
TokenWatcher was built using a modern full-stack architecture.
Frontend
- React
- Vite
- TypeScript
- Tailwind CSS
- shadcn/ui
- React Query
Backend
- Node.js
- Express
- PostgreSQL
- Server-Sent Events for real-time updates
Developer SDK
- TypeScript SDK with batching, retries, queue management, and graceful shutdown.
Integrations
- Telegram Bot API
- OpenClaw
- AI-powered insights and recommendations
Throughout development, we continuously refined the architecture, improved reliability, and optimized the developer experience to create a production-ready observability platform.
Challenges we ran into
Building a real-time observability platform presented several challenges.
Some of the biggest were:
- Designing a reliable telemetry ingestion pipeline
- Managing workspace isolation securely
- Handling large volumes of telemetry efficiently
- Building responsive real-time dashboards
- Creating meaningful analytics instead of simply displaying raw metrics
- Integrating conversational Telegram workflows while maintaining security
Balancing performance, scalability, and usability required several iterations throughout development.
Accomplishments that we're proud of
We're especially proud that TokenWatcher evolved beyond a simple dashboard into a complete observability platform.
Highlights include:
- Real-time telemetry dashboard
- Multi-workspace architecture
- Production-ready TypeScript SDK
- AI-powered recommendations and forecasting
- Live analytics using Server-Sent Events
- Telegram integration for remote monitoring
- Comprehensive documentation and deployment guides
- A clean, developer-focused user experience
What we learned
Building TokenWatcher reinforced that AI infrastructure needs observability just as much as traditional software systems.
We learned how important it is to design reliable telemetry pipelines, build scalable analytics systems, and create developer tools that are easy to integrate while remaining production-ready.
The project also deepened our understanding of backend architecture, real-time systems, SDK design, and AI infrastructure.
What's next for TokenWatcher – AI Cost & Observability Platform
Our long-term vision is to make TokenWatcher a complete AI observability platform for engineering teams.
Planned improvements include:
- Support for additional AI providers
- Advanced anomaly detection
- Smarter forecasting models
- More conversational AI workflows
- Team collaboration features
- Usage budgets and governance policies
- Enterprise deployment options
- Expanded reporting and export capabilities
Our goal is to help developers build AI applications with greater visibility, reliability, and cost efficiency.
Log in or sign up for Devpost to join the conversation.