GeminiLens: Gemini Agent Observability on TrueFoundry

• 

  GeminiLens at a glance: four resilience signals shipped to TrueFoundry as OTLP spans.

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

Inspiration

Every "resilient agent" we've shipped started life as a fragile one. Not because we didn't write retries, but because we couldn't see the cost climbing, the p95 latency drifting, or our tool calls quietly hitting hosts we never approved. So we built the observability layer first, then realized it's the resilience layer in disguise.

What it does

GeminiLens is a small Python library plus a Streamlit dashboard that wraps every Gemini call and emits a Trace with four resilience signals:

• Cost spike: per-call USD cost with cached-input pricing. Catches prompt regressions that double your bill.
• Latency drift: rolling p95 vs baseline. Catches Gemini slowdowns or retrieval bloat.
• Error rate: every Gemini call wrapped, errors tagged in the trace.
• Egress violation: tool calls go through an httpx allowlist transport; reach the wrong host, you get an exception, not a quiet exfiltration.

The new TrueFoundryExporter ships those traces to your TrueFoundry AI Gateway as OTLP/HTTP-JSON spans with Gen-AI semantic-convention attributes (gen_ai.usage.cost_usd, input_tokens, output_tokens, error status). Alert on cost spikes, latency drift, error bursts, or egress allowlist violations inside the platform you already pay for.

GeminiLens is the eyes. TrueFoundry is the brain that acts on what it sees.

How we built it

• google-genai + Vertex AI Gemini 2.5 for the agent calls
• httpx custom transport for the egress allowlist
• Streamlit + pandas for the live dashboard
• Pure stdlib for cost math and drift, so nothing has to be mocked in tests
• TrueFoundryExporter (the new piece for this submission) ships OTLP/HTTP-JSON to /api/otel/v1/traces with Bearer auth and the TFY-Tracing-Project header
• Test suite: 5 tests on the TrueFoundry exporter cover credential validation, OTLP shape, Gen-AI semconv attrs, error status mapping, batched export. 3 tests on the existing Dynatrace exporter still pass.

Challenges we ran into

Resilience signals need stable baselines. With small trace counts, "p95 doubled" is just noise. We split baseline vs rolling windows explicitly so reviewers can see the sample counts on the dashboard. Also: google-genai's usage_metadata shape varies between client versions and between Vertex AI and the API; the observer handles both.

Accomplishments that we're proud of

Zero new dependencies for the TrueFoundry integration (httpx was already core). The TrueFoundryExporter is 130 lines of code and 110 lines of tests. Reviewers can audit it in five minutes. The cost table is hand-curated from Google's published Gemini pricing and lives in the repo, so reviewers can verify the pricing source.

What we learned

Resilience for LLM agents is more about visibility than about retries. The hardest failure mode is not "Gemini returned an error", it's "Gemini returned a longer, more expensive, slightly worse answer and your eval missed it." Observability with Gen-AI semconv attributes makes that detectable on day one.

What's next for GeminiLens

• A TrueFoundry-side recipe that turns the OTLP spans into auto-degradation: if 5xx errors exceed 1% for 5 minutes, fall back to gemini-2.5-flash from gemini-2.5-pro
• Span linking so multi-step agent runs render as one trace tree in TrueFoundry
• A Sigstore-signed cost table so reviewers can verify the pricing source
• More exporters: Phoenix, Langfuse, Honeycomb ## What we learned

What's next for GeminiLens: Gemini Agent Observability on TrueFoundry

Built With

• gemini
• google-cloud
• httpx
• observability
• opentelemetry
• pandas
• python
• streamlit
• truefoundry
• vertex-ai