Retina

• 

  logo

• 

  in action

• 

  agent thought process

• 

  settings for person

• 

  ui

Inspiration

My aunt was born blind. Growing up with her she would always be interested in the tech space of things - but her inability to interact with these machines made it hard to learn and use them. We made Retina so the blind aren't limited to themselves

What it does

Retina is a voice only AI powered partner that can understand your needs without having to touch your device. It coordinates five specialized agents:

1. A general agent for questions, research, and web navigation.
2. A commerce agent for finding and comparing products.
3. A coding agent for working with files, terminals, and codebases.
4. A desktop agent for opening and controlling applications such as Spotify and Visual Studio Code.
   1. A documentation agent for creating taking down information.

How we built it

We built Retina as an Electron desktop application using React, TypeScript, and Vite.

A Node.js backend communicates with the interface through Socket.IO. Gemini handles speech recognition and voice generation, while Claude powers intent classification, reasoning, and desktop interaction. LangGraph routes each request to the correct specialist agent, and Stagehand provides browser automation.

For desktop control, Retina combines AI computer vision with direct macOS automation. Straightforward actions—such as opening Spotify— use deterministic system commands for speed and reliability. More complex interactions use screenshots, keyboard input, and mouse control.

Challenges we ran into

Efficiency, Voice Input, Browser/Desktop navigation, Blind constraints

Accomplishments that we're proud of

We could take a real problem and produce a product to solve it

What we learned

Most things don't working production /// tons of testing - building makes for a good product

What's next for Retina

Improving desktop workflows and adding an IOS app

Built With

• codex