We strongly encourage you to go through our slides - contains great external information not included in the video.
Youtube Link 📹 : https://www.youtube.com/watch?v=iWQf-PtMEqU
❓ What is our project?
Our project is a hand detector that uses computer vision. Hand_AI detects a person's hand and a specific hand symbol they are holding up (can be anything really). The computer has specific responses to each symbol meaning it will output a specific action, so if you hold up 1 finger it outputs action 1, holding 2 fingers outputs action 2, etc. To put our idea into perspective, a user may give a thumbs up to the camera triggering a door to open, as shown in our demo.
💻 How did we build the product/code?
Using Python and OpenCV, we are able to make a successful detection program. By implementing Machine Learning and Deep Learning algorithms, we are able to make this program quick, efficient, and accurate. Our program utilizes these algorithms by first extracting the shape of the person’s hand provided by the ROI. The ROI is a rectangle where we ask the person to hold their hands in. After we extract the shape that the person’s hand is in, we then perform masking as well as different calculations to calculate what pose the hand is performing. The poses we can estimate are the numbers 0-5, “Thumbs up”, and the “Ok” gesture. In our case, we decided to take the corresponding action with the “Thumbs up” hand gesture simply because we believed that it was the most logical gesture that indicates that the person is ready to do something. To open the door, we use a motor(connected to the Raspberry Pi’s GPIO input pins), and a rope that is attached to the door handle. When the motor rotates, the door is pulled open. In our program, once the gesture “thumbs up” is detected, we turn the GPIO signal connected to the motor on. This act of “flipping the switch” turns on the motor until we turn the signal back off, which we do after 5 seconds (the time it takes for the motor to pull open the door). So far, this is all we have done, but in the future, more actions can be done with more hand signals (person holding number 1 -> conveyor belt sends item number 1 to the individual).
🔧 Hardware components used?
For the overall product: 1. Raspberry Pi 2. Raspberry Pi Camera 3. Any accessory that needs to be turned on/off with different hand signals. In our hackathon: 1. We added a DC motor that opens the door.
😢 Problems Faced? It wouldn’t be a hackathon without some problems!
It took us a lot of time to actually come up with an idea to solve. Our code was very inconsistent at times - sometimes it worked, sometimes it didn’t. We ended up fixing this problem however this especially was something that really frustrated us. One of our team members had an urgent matter to attend to and could not attend the rest of the hackathon - the rest of us had to pick up the slack and work extra hard.
✅ Problems Solved in the World?
Just a few of the problems that we solved by creating this product include on-going issues and current events. Starting off with on-going issues, our product makes it possible so if someone wants to signify that they are crossing the road, instead of pressing the button to alert the system, you can raise your finger up to the camera and firstly it makes the process much more quickly and accurately, and secondly, it is much more sanitary because the button that was originally used was never clean or disinfected and this product completely eliminates the concern with that. This also tackles current issues because with COVID-19 you want to avoid as many germs and touching objects as possible and this helps with that. Another one of the many problems this product solves is helping people open doors so they can lift up a finger (1) and the door will open automatically and hold up a number (2) if they would like to close the door. This eliminates all the contact that was previously required and makes life much easier for people who may be handicapped or are holding many things in their hands.
📆 Plans for the future?
For the future of this product, we hope to officially implement it to help make streets and other places more sanitary and much more efficient when it comes to signifying when you're crossing the road, to help handicapped people open doors, etc. Also, we hope to add more abilities to our hand-raising feature with fingers including utilizing this tool in online school to help students and teachers make the online teaching and learning experience much easier, by detecting the child's hand and then alerting the teacher that the student is raising their hand. This tool can be used in various places to help for more efficiency and overall its price is also extremely cheap making it affordable and not only being limited to the wealthy. We would also like to further improve our product design in order to make it more accessible to the general public, making the future of tomorrow in the grasp of our hands.