AirFlow

Inspiration

We just took the theme literally. "break the norm? ok, so im not gonna use my computer like its designed." we later noticed that this was useful for people who like to get their hands dirty, mechanics, bakers, cooks. those who need to check something on their computer without ruining it or disrupting their flow.

What it does

We utilize both hand gestures and verbal queues to navigate your computer. Using your camera and Google's MediaPipe, and Gemma 4, Vultr, and Eleven labs for verbal controlled Assistant. A way to control your computer, Hands free. The ability to flow through your computer, while keeping distance.

How we built it

The app was built as an Electron desktop app with a React/TypeScript frontend. It uses the device camera with Google MediaPipe to recognize hand gestures, then maps those gestures to computer controls. Voice control is powered by AI services including Gemma, Vultr, and ElevenLabs, allowing users to control their computer through spoken commands as well.

Challenges we ran into

Using Gemma 4 to navigate the computer proved difficult and resource-heavy, especially alongside real-time gesture control. Offloading language parsing and intent decision-making to Vultr helped reduce local overhead and made the system more responsive.

Accomplishments that we're proud of

Ariston and Jake developing the first glimpse of cursor movement was novelty! Later developing 6-7 new gestures. Nathan developed a voice activated AI, allowing the ability to google search and navigate through links. while also being able to open apps on your computer and searching for music.

What we learned

We learned collectively how to use new technologies on the fly. Gemma 4, Vultr, ElevenLabs, Electron, Google MediaPipe, the list goes on...

What's next for AirFlow

next is better integration of the voice activated AI. More gestures and the ability to make macros for gestures. And potentially running the entire system through Vultr to keep it light for other hardware.

Built With

• electron
• elevenlabs
• gemma4
• mediapipe
• nut-js
• react
• typescript
• vultr
• zustand