ARIA

An autonomous research agent that plans, executes, critiques itself, and replans — and you watch the loop run live.

Comment

Inspiration

Most "AI research assistants" are a single LLM call wearing a trench coat: one prompt, one answer, no idea whether the answer is any good. Real research isn't one step — it's plan, look things up, judge what you found, go back for what's weak, then write it up.

I wanted to build that loop as an actual state machine — not a chatbot — and make it watchable: every plan, search, score, and replan streaming live as it happens.

What it does

You give ARIA a research question. It:

  1. Plans — breaks the question into subtasks.
  2. Executes — searches four sources in parallel: live web (Tavily), arXiv, Wikipedia, and GitHub. Findings are deduplicated by URL, ranked, and source-attributed ([Web], [arXiv]…).
  3. Critiques itself — each finding is scored 0–10 across four dimensions (relevance, specificity, source quality, completeness) using structured Pydantic output, not regex-parsing of prose.
  4. Replans surgically — if a subtask scores below 7, ARIA replans only that subtask (capped at 2 replans), instead of throwing away the whole run.
  5. Reports — a structured, multi-section report with inline sources and one-click PDF export. Every run is saved to resumable session history.

The Streamlit UI is a monospace "blueprint" console where you watch the cognitive loop run in real time — executor searches, critic scores, and replans appear live with a progress bar. Try it: agent-aria.streamlit.app.

How I built it

  • LangGraph state machine: Memory Reader → Planner → Executor → Critic → [Replan?] → Memory Writer → Terminator → Report. Nodes share a typed AgentState (TypedDict).
  • A custom state reducer: LangGraph merges node outputs, and the naive approach duplicates findings on every replan. I wrote a merge_results reducer that keys findings by a stable ID — so a replanned subtask's scores update in place instead of appending duplicates.
  • Structured critic: the critic returns a Pydantic model, so scores are typed numbers with hard bounds — no "the model said approximately 8/10" string parsing.
  • Parallel multi-source search with per-source attribution, and sources toggleable per run.
  • Groq for inference (fast + free tier), Tavily for live web search; arXiv and Wikipedia need no keys.
  • Engineering hygiene: 32 passing tests, CI on GitHub Actions, MIT license, pinned dependencies, committed example runs in examples/ so the results are reproducible.

Challenges I ran into

  • Replan loops love to run forever. An agent that criticizes itself will happily replan into infinity. The fix was two-fold: a hard max_replans cap and making replanning targeted — one weak subtask, not the whole plan.
  • State merging. Getting LangGraph to update a finding rather than duplicate it took a custom reducer and a stable ID scheme — the least glamorous, most important code in the repo.
  • Judging the judge. The critic's scores are only useful if they're consistent; a four-dimension rubric with numeric bounds made scores stable enough to act on, where a single "how good is this?" score wasn't.
  • Making an agent watchable. Streaming internal state to a UI mid-run without blocking the graph took real plumbing — but it's the difference between "trust me, it's thinking" and literally watching it think.

Accomplishments that I'm proud of

  • A genuinely autonomous loop — plan, search, self-score, targeted replan — that generalizes: no hardcoded knowledge, every finding is live search on any topic you type.
  • The live console UI. People don't believe agent demos anymore; ARIA shows its work.
  • 32 tests and CI on an agent project — most agent repos have zero.
  • Hand-built, not scaffolded: the graph, the reducer, the critic rubric are all mine.

What I learned

  • Agent quality lives in the control flow, not the prompts. The critic threshold, the replan cap, and the state reducer shaped output quality more than any prompt wording.
  • Structured output (Pydantic) is the difference between an agent you can build logic on and one you babysit.
  • Observability is a feature. Making the loop visible changed how I debugged it — and how much people trust it.

What's next for ARIA

  • Evals and benchmarks against Perplexity-style baselines, published in the repo.
  • Filling the committed results table with more standard runs (scripts/run_examples.py).
  • More sources (Semantic Scholar, news APIs) behind the same driver interface.
  • Memory across sessions — letting ARIA build on its own previous research.

Built by Aarti Panchal — GitHub @Aarti-panchal01

Built With

Share this project:

Updates