Inspiration
The idea for Space Buttons came in following scenario: I was enjoying a meal, hands messy from food, and suddenly needed to interact with my computer. The problem? My greasy fingers were in no shape to touch the keyboard or mouse, and cleaning up every time I wanted to scroll or change a tab disrupted the whole experience. I realized that navigating the web should be as seamless as enjoying a meal — hands-free and hassle-free. That's when I started thinking: what if there was a way to control the screen with simple gestures, allowing me to continue enjoying my food without the interruptions? Hence, the birth of Space Buttons, the perfect tool for hands-free web browsing!
What it does
Space Buttons allows you to navigate the web using simple hand gestures. There are five key gestures that perform specific functions:
Navigate – Move through pages easily. Scroll Up/Down – Effortlessly browse long pages. Volume Control – Adjust audio without leaving your seat. Click – Interact with pages via gestures. No need for a mouse or keyboard – just wave your hand and you're good to go!
How I built it
The project leverages computer vision and hand tracking to perform various tasks on the computer using hand gestures. We start by capturing live video feed using OpenCV and configuring the camera settings. A custom-built HandTrackingModule detects hand landmarks, which are used to track the position of fingers.
Based on the hand positions, specific gestures are mapped to control system actions:
Scroll Mode: When the user shows a specific finger gesture (index finger up), scrolling is activated. A gesture with two fingers triggers downward scrolling using pyautogui. Volume Mode: The distance between the thumb and index finger is used to control system volume. The distance is converted to a volume level using numpy and set via the pycaw library. Cursor Mode: The index finger controls the cursor movement, while making a fist (thumb down) triggers mouse clicks. The cursor's position is adjusted using autopy. Finally, the code calculates and displays FPS to track performance, with a user interface built using OpenCV to show the active mode and feedback on gesture actions.
This combination of libraries — OpenCV for hand tracking, PyAutoGUI and Autopy for system control, and Pycaw for volume — brings together a hands-free web navigation solution.
I know the hand gestures seems complicated to understand through text, so I reccomend you to watch the video for clear explaination.
Challenges I ran into
The main challenge was integrating the backend hand detection logic, written in Python, with the frontend in a way that allowed seamless functionality on the web. As a solo developer, time was a constraint, especially when trying to build an API to enable users to access the features without needing to download and run the code locally.
Accomplishments that I'm proud of
Despite the time crunch, I successfully implemented a functional frontend that integrates Python hand detection for gesture-based web navigation. The overall utility works smoothly, and the minimalistic web design allows for an intuitive user experience.
What I learned
Through this project, I learned how to effectively use Python libraries like OpenCV for hand detection, improving my understanding of how to connect computer vision with everyday tasks. I also gained valuable experience in frontend design, bridging the gap between visual appeal and technical functionality.
What's next for Space Buttons
The next step is to improve the backend integration. By creating an API, users will be able to grant permission and use Space Buttons directly from their browser without needing to run the code manually. This will make the feature more accessible and user-friendly, allowing for greater distribution and adoption of hands-free web navigation.
Log in or sign up for Devpost to join the conversation.