Inspiration
Tired of clicking through interfaces — we wanted an AI that acts for you, not just answers.
What it does
ARIA is a voice + vision browser agent. You speak, it sees your screen, and autonomously navigates, clicks, types, and completes tasks in real-time. Interrupt anytime and it pivots naturally.
How we built it
- Gemini Live API — real-time voice streaming
- Gemini Vision — screenshot understanding every 2s
- Google ADK — agent orchestration with 14+ browser tools
- Playwright — browser automation
- FastAPI + React — backend/frontend, deployed on Cloud Run ##Challenges we ran into
- Syncing audio streaming with vision context without race conditions
- Clean interruption mid-task without breaking browser state
- Routing "do this" commands vs. "answer this" questions reliably ##Accomplishments that we're proud of
- Sub-second voice response latency
- Working mid-task interruption ("actually, stop")
- Isolated sessions per user, one-command local setup ##What we learned Gemini Live removes most audio pipeline complexity. ADK is powerful but needs careful prompt tuning to avoid redundant tool calls. Multimodal boundaries (PCM, PNG, JSON) matter a lot. ##What's next for ARIA Multi-tab orchestration, persistent memory, mobile support, and proactive screen monitoring.
Log in or sign up for Devpost to join the conversation.