EXCLAVE

• 

  Website

• 

  Logo

Overview:

Our project is a gesture-based authentication system designed to automate the login process and make access to websites more accessible for a wider range of users. Instead of requiring someone to type a password every time, our system uses machine learning to recognize a combination of hand gestures, object detection, and object color through a webcam. Each unique combination is paired with a specific website, so users can unlock different sites based on the gesture and object sequence they present. This creates a more interactive, touch-free, and personalized alternative to traditional login methods. Our goal was to build something that felt both practical and innovative while also connecting to the hackathon theme of automation.

Inspiration:

Our inspiration for this project came from wanting to make technology easier for more people and create something that could have a real impact in everyday life. We wanted to build a system that would be useful not only for regular users, but also for people with disabilities who may have difficulty typing or using traditional login methods. By allowing users to authenticate with gestures and objects instead of only passwords, we hoped to make access more flexible and inclusive. We were especially motivated by the idea of creating something that could improve accessibility while still being practical for normal daily use. This focus on usability and inclusion guided the design of our project and helped us build something that could make everyday interactions with technology simpler. In the end, our project lets all of this happen by combining accessibility, automation, and machine learning into one system.

What It Does:

The system works by continuously analyzing live webcam input and identifying both the object in view and the hand gesture being made. For example, a user might hold up a specific object of a certain color and then perform a gesture such as showing the number three with their hand. The machine learning models detect these inputs in real time and compare them to the combinations that have been registered in the system. If the combination matches one that has been paired with a website, the user is granted access to that site automatically. This allows different gesture-object combinations to function like unique keys for different websites, making the experience more flexible and customizable.

How We Built It:

To build the project, we combined several existing technologies into one working prototype. We used Google’s MediaPipe hand landmark detection model to identify hand gestures and basic object recognition tools to detect objects and their colors through the webcam. We then incorporated these models into a mock login website that simulates a computer lock screen and acts as the interface for the authentication process. The website was built using HTML and Python, and we linked the machine learning components so they could communicate with the front end in real time. By connecting the webcam feed to the detection models, we were able to create a system that responds instantly to user input and routes them to the correct website based on the recognized combination.

Challenges We Faced:

One of the biggest challenges we faced was how close we got to the deadline while still trying to make everything work smoothly. Getting all of the different pieces to communicate with each other took much longer than expected, especially when linking the webcam input, gesture detection, object recognition, and website navigation together. Sharing data between the different parts of the project also took a lot of time because each component had to be carefully connected and tested. We also had to spend time troubleshooting small issues in the detection process to make sure the system could reliably recognize the correct combinations. Even though the process was difficult, it pushed us to stay organized and keep refining the project until everything came together.

What We Learned:

Through this project, we learned a lot about machine learning, Python, and HTML, as well as how to combine them into one functional system. We gained a better understanding of how existing AI tools like MediaPipe can be incorporated into real-world applications rather than being used on their own. We also learned how important it is to structure a project carefully when multiple technologies need to work together. In addition, we improved our problem-solving and teamwork skills by figuring out how to debug issues, connect different systems, and manage our time effectively. Overall, this project taught us how automation and machine learning can be used together to create more accessible and user-friendly technology.

Future Improvements:

In the future, we hope to expand the system by adding more complex hand gestures and longer gesture sequences to make authentication even more secure and customizable. We also want to improve the accuracy of object and color detection so the system can recognize combinations more reliably in different lighting conditions. Another possible improvement would be allowing users to create and save their own gesture-object combinations for different websites. With more development, this project could become a more advanced and practical alternative to traditional login methods while still keeping accessibility at the center of the design.

Built With

• html
• python