Flippers

Real-time research-powered extension for TerpAI

Comment

Inspiration

TerpAI can hand off structured “research requests,” but turning those into verified, reviewable evidence (especially when URLs, dining menus, nutrition data, or campus events are involved) still takes glue code and manual checking. Flippers was inspired by the idea of a small companion that sits beside TerpAI: it listens for those handoffs, runs a local pipeline to gather and synthesize evidence, and puts you in the loop before anything is sent back—so “Testudo gets backup” without losing control of what gets submitted.

What it does

Flippers is a Chrome extension (Manifest V3) that injects an overlay on TerpAI chat pages. It watches for TerpAI’s plain-text TERPAI_RESEARCH_REQUEST protocol, sends that text to a FastAPI backend, and drives a LangGraph workflow: parse → gather → synthesize, with Google Gemini and optional Playwright-backed lookups. State is persisted in SQLite (app DB + LangGraph checkpoints). The extension shows status in the UI and, when evidence is ready, lets the user review and approve before completing the TerpAI request via the API. Supported flows include general research, URL evidence lookup, healthy-terp menu/nutrition lookups, and hungry-terp event lookup—see backend/app/protocol.py for command types.

How we built it

  1. Extension: TypeScript, esbuild bundling into extension/dist/, background service worker + content script, overlay (extension/src/overlay/) with states like listening, researching, review, and error; TerpAI DOM helpers for observing chat and completing requests.

  2. Backend: FastAPI (uvicorn), SQLAlchemy models for cycles/requests, LangGraph graph with SQLite checkpoints, Gemini client (GEMINI_API_KEY / config in app/config.py).

  3. Pipelines: backend/pipelines/ implements Playwright-based extraction (e.g. menu, nutrition, event lookups) registered and invoked from the gather stage.

  4. Protocol: Deterministic parser for TERPAI_RESEARCH_REQUEST blocks with fields like command_type, goal, and candidate_urls.

Challenges we ran into

  1. Stable TerpAI integration: Keeping the overlay and observers in sync with TerpAI’s chat DOM and avoiding duplicate handling of the same request_id.

  2. Ops for Playwright: Installing Chromium binaries and making pipelines reliable across varied sites.

Accomplishments that we're proud of

  1. Pluggable command types and a plain-text protocol that the backend parses deterministically.

  2. Durable workflow state via LangGraph checkpoints and a proper cycle/request model in SQLite.

What we learned

  1. Playwright as a practical way to turn “look at this URL / this dining page” into structured evidence—along with the maintenance cost of site-specific logic.

  2. The value of an explicit protocol between the assistant UI and your own automation layer.

What's next for Flippers

Richer pipelines and evaluation for extraction quality and failure modes.

Built With

Share this project:

Updates