Inspiration

We interact with our computers on a daily through mice and keyboards, but what if we could interact with desktop windows like physical objects?

That is the exact question which inspired W.A.V.E, a touchless Windows desktop controller that turns a simple webcam into a new way of interacting with your computer. We wanted to make desktop interactions more intuitive, creative, and personalized without requiring any specialized hardware.

What it does

W.A.V.E uses hand gestures to control real Windows applications. Users can pinch to grab and move windows, snap them to screen edges, scroll with a peace sign, and switch applications with a simple swipe.

Our standout feature is Wave Scenes. Users can create personalized, application-specific workspace layouts and restore them with a single hand gesture. Whether you're coding, researching, or multitasking, your whole workspace can return to its preferred arrangement almost instantly.

How we built it

We developed W.A.V.E using Angular 18 and TypeScript for the frontend and ASP.NET Core (.NET 8) for the backend.

Google's MediaPipe Hand Landmarker tracks 21 hand landmarks locally through the webcam. Our gesture recognition system interprets these movements, while the backend communicates with native Windows APIs to manipulate real application windows.

We also implemented cursor smoothing, gesture state management, and configurable recognition thresholds to improve reliability.

Challenges we ran into

One of our biggest challenges was translating natural hand movements into precise desktop actions. Small movements could cause jitter, while similar gestures sometimes conflicted.

We addressed these issues through smoothing filters, gesture-specific thresholds, and cooldown mechanisms.

Another challenge was controlling actual Windows applications rather than simulated browser elements, which required accounting for window borders, screen scaling, and positioning.

Accomplishments that we're proud of

Our proudest accomplishment is turning an unconventional idea into a working application that controls real Windows applications, not just simulated browser elements.

We're especially proud of developing Wave Scenes, which lets users create personalized workspaces and restore multiple applications with a single hand gesture.

We also successfully integrated real-time computer vision with native Windows APIs, implemented multiple distinct gestures and kept hand tracking entirely local, all using a simple webcam.

More importantly though, we built something that challenges how people traditionally interact with their computers.

What we learned

We learned how to combine computer vision, real-time gesture recognition, and native operating system APIs into one functional application.

More importantly, we discovered that creativity isn't always about inventing new technology, sometimes, it's about combining existing technologies to rethink everyday interactions.

What's next for W.A.V.E

We hope to expand gesture customization and explore accessibility-focused applications. Our vision is simple: Your desktop. Your gestures. Your W.A.V.E

Built With

Share this project:

Updates

Submission history