Flow | Devpost

Inspiration

It started when I noticed my grandma’s hands trembling. She can no longer hold or use her phone, and watching her struggle sparked an idea—technology should adapt to people, not the other way around. Flow was built to give her, and anyone with limited hand mobility, the same freedom to use a computer as everyone else.

What it does

Flow enables complete hands-free computer control.

Facial gestures move the mouse and trigger clicks or drags.
Voice commands let the user type, dictate messages, send emails, or ask an AI agent questions like “What’s the weather?”—all without touching a keyboard or mouse.

How we built it

Real-time face landmarking and calibration provide precise facial control.
LangGraph powers a multi-agent system for complex actions.
Airia handles multi-agent coordination and MCP (Model Context Protocol).
Gladia provides low-latency speech-to-text.
Minimax generates natural text-to-speech responses.
Bright Data enables WebMCP integration for external data retrieval.

Challenges we ran into

Calibrating facial structure to avoid mis-cues and achieve smooth, accurate cursor control.
Reducing STT and TTS latency, which we solved with streaming and intelligent end-of-speech detection.
Designing reliable ways to bring up the AI agent exactly when the user starts speaking.
Creating intuitive toggles for different agent modes, such as asking a question versus dictating text.

Accomplishments that we're proud of

We achieved fast, accurate face-controlled navigation and a responsive multi-agent voice interface that runs on everyday laptops without special hardware. Seeing my grandma successfully use a computer again with only her face and voice was the ultimate proof of impact.

What we learned

Start simple and get a working prototype before adding complexity.
Multi-agent systems add power but also increase build complexity and latency, which must be carefully managed.
Facial calibration is far harder in practice than it appears on paper.

What's next for Flow

Our next step is to refine the current experience, release it to real users, and gather feedback to guide updates. We’ll focus on stability, user comfort, and the features people value most before expanding to new platforms and integrations.