Glassbox Agent

Multi-agent observability copilot that plans SQL + diagnostics runs, routes metrics/chart agents, and explains your telemetry end‑to‑end

Comment

📘 Project Story: Infinite Loop

⭐ About the Project

Infinite Loop was built to solve a real pain point for agentic-AI engineers and researchers:
LLM agents produce long, complex execution traces — but current tools make it hard to understand why an agent behaved the way it did.

Modern agents loop, branch, call tools, spawn sub-agents, and sometimes exhibit what researchers call emergent misalignment. Yet the underlying telemetry (LangSmith traces) is nested JSON that is difficult to query or interpret.

Infinite Loop provides the observability and interpretability layer missing in today’s agentic ecosystems.

🌱 What Inspired Us

Three forces motivated the project:

  1. Engineering frustration
    Teams repeatedly said:
    “I know the agent is misbehaving, but I can’t see why.”

  2. Multi-agent research challenges
    Researchers studying coordination failures lacked a structured way to analyze thousands of traces.
    We wanted to help answer questions like: [ \text{What patterns predict cascading tool errors in multi-agent rollouts?} ]

  3. The rise of natural-language analytics
    If SQL can express observability queries, then an LLM can translate human questions into analytics operations.

🏗️ How We Built It

1. Trace Normalization

We ingest raw LangSmith traces and convert them into a clean relational schema:

  • agent_runs: root-level metadata
  • steps: every LLM/tool/chain call
  • normalized token + cost + timing fields
  • reconstructed execution chains using: [ \text{previous_step_id} \rightarrow \text{step_id} ]

2. Natural Language → Intent → SQL

Users ask:

“Which tool had the highest latency yesterday?”

The observability agent:

  1. Parses into structured intent
  2. Generates safe SQL
  3. Executes it
  4. Cleans the output
  5. Summarizes results in readable form

3. Robust Failure Handling

We implemented:

  • automatic handling of missing fields
  • timestamp coercion
  • fallback summaries for empty results
  • error-aware SQL execution

🧠 What We Learned

1. Agent debugging requires causal visibility

Dashboards aren't enough. Engineers need step-level insight into chains of reasoning.

2. Natural language is a powerful debugging interface

Asking: [ \text{“What caused the loop?”} ] and receiving a precise answer changes the workflow entirely.

3. Relational schemas enable interpretability

Flattening traces revealed bottlenecks, hidden loops, misaligned sub-agents, and long-running tool sequences.

🔧 Challenges We Faced

  • Reconstructing run hierarchies from parent_run_id
  • Designing a schema flexible enough for diverse agent behaviors
  • Ensuring safe, reliable SQL generation
  • Handling inconsistent timestamp formats: [ \text{elapsed_seconds} = t_{\text{end}} - t_{\text{start}} ]

🚀 Final Thoughts

Infinite Loop is an interpretability assistant for agentic AI.
It helps engineers and researchers see inside the systems they build — understand failures, trace multi-step reasoning, and monitor complex behaviors.

If you work with LLM agents, tool-calling pipelines, or multi-agent systems, Infinite Loop is built for you.

Built With

Share this project:

Updates