ObservAI - LLM Observability and Prompt Optimizer

LLMs fail silently. ObservAI makes them observable. Track cost, latency, hallucinations, and quality for Gemini in real time — with a single drop-in SDK. Ship AI with confidence, not guesswork.

Comment

💡 Inspiration

As AI systems increasingly power global products — from cross-border hiring platforms to international collaboration tools — we realized something critical was missing:

Teams have almost no visibility into how AI behaves differently across regions.

The same AI model can:

  • cost significantly more in one country than another,
  • hallucinate or degrade in quality for certain regions,
  • silently violate compliance or fairness expectations,
  • fail without any clear audit trail.

Inspired by VisaVerse’s vision of borderless opportunity and global mobility, we built ObservAI to act as a trust layer for global AI systems — making AI decisions transparent, auditable, and fair no matter where users are located.

🚀 What ObservAI Does

ObservAI is an AI observability and governance platform that monitors every AI interaction across regions in real time.

It helps teams:

  • 📊 track AI performance (latency, failures)
  • 💰 understand AI cost per region
  • 🧠 detect hallucination, toxicity, and quality risks
  • 🌍 audit AI behavior across borders
  • 🚨 receive real-time alerts when AI systems misbehave

All of this works as a drop-in SDK for Gemini / Vertex AI, with Datadog-powered observability under the hood.

🛠️ How We Built It

ObservAI was built using a production-grade observability stack designed for modern AI systems.

Architecture Overview

$$ \text{User Application} \;\rightarrow\; \text{ObservAI SDK powered by Gemini 3 Flash on Vertex AI} \;\rightarrow\; \text{Antigravity OpenTelemetry Layer} \;\rightarrow\; \text{Datadog Observability Platform} \;\rightarrow\; \text{ObservAI Insights & Governance Dashboard} $$

Core Technologies

  • AI Model: Gemini 3 Flash
  • Instrumentation: Antigravity + OpenTelemetry (OTLP)
  • Observability Platform: Datadog
  • Backend: Supabase (Edge Functions + PostgreSQL)
  • Frontend & Hosting: Vercel
  • Language: TypeScript

Each AI request is automatically enriched with metadata like:

  • region
  • latency
  • token usage
  • cost
  • hallucination risk
  • safety signals

These signals are streamed to Datadog, where we define custom dashboards, detection rules, and alerts.

🚧 Challenges We Faced

  • AI-specific observability is fundamentally different from traditional infrastructure monitoring
  • Capturing hallucination and safety signals required custom logic beyond standard metrics
  • Making observability work reliably in serverless production environments
  • Designing alerts that are actionable instead of noisy
  • Mapping AI behavior to real-world cross-border risks

We overcame these by treating AI as a first-class risk system, not just another API call.

📚 What We Learned

  • Traditional APM tools alone are not enough for AI systems
  • AI observability must include quality, safety, and fairness signals
  • Developers want tools that are invisible, automatic, and low-friction
  • When combined with Datadog, AI telemetry becomes deeply actionable
  • Global AI products need auditability by default, not as an afterthought

🌱 Impact & Future Vision

ObservAI enables teams to:

  • build safer global AI products
  • ensure fairness across regions
  • reduce AI costs intelligently
  • comply with emerging AI regulations
  • trust AI systems at scale

Our long-term vision is to make ObservAI the trust infrastructure for global AI, helping people work, move, and collaborate across borders — safely and transparently.

Built With

Share this project:

Updates