1.3 billion People have some sort of vision impairment. They face difficulties in simple day to day task like reading, recognizing faces, objects, etc. Despite the huge number surprisingly there are only a few devices in the market to aid them, which can be hard on the pocket (5000$ - 10000$!!). These devices essentially just magnify the images and only help those with mild to moderate impairment. There is no product in circulation for those who are completely blind.
What it does
The Third Eye brings a plethora of features at just 5% the cost. We set our minds to come up with a device that provides much more than just a sense of sight and most importantly is affordable to all. We see this product as an edge cutting technology for futuristic development of Assistive technologies.
Guidance - Uses haptic feedback to navigate the user to the room they choose avoiding all obstacles. In fact, it's a soothing robotic head massage guiding you through the obstacles around you. Believe me, you're going to love it.
Home Automation - Provides full control over all house appliances. With our device, you can just call out Alexa or even using the mobile app and tell it to switch off those appliances directly from the bed now.
Face Recognition - Recognize friends (even their emotions (P.S: Thanks to Cloud Vision's accurate facial recognitions!). Found someone new? Don't worry, on your command, we register their face in our database to ensure the next meeting is no more anonymous and awkward!
Event Description - Describes the activity taking place. A group of people waving somewhere and you still not sure what's going on next to you? Fear not, we have made this device very much alive as this specific feature gives speech feedback describing the scenic beauty around you. [Thanks to Microsoft Azure API]
Read Up - You don't need to spend some extra bucks for blind based products like braille devices. Whether it be general printed text or a handwritten note. With the help of Google Cloud Vision, we got you covered from both ends. Read up not only decodes the text from the image but using Google text to speech, we also convert the decoded data into a speech so that the blind person won't face any difficulty reading any kind of books or notes they want.
Object Locator - Okay, so whether we are blind or not, we all have this bad habit of misplacing things. Even with the two eyes, sometimes it's too much pain to find the misplaced things in our rooms. And so, we have added the feature of locating most generic objects within the camera frame with its approximate location. You can either ask for a specific object which you're looking for or just get the feedback of all the objects Google Cloud Vision has found for you.
Text-a-Friend - In the world full of virtuality and social media, we can be pushed back if we don't have access to the fully connected online world. Typing could be difficult at times if you have vision issues and so using Twilio API now you can easily send text messages to saved contacts.
SOS - Okay, so I am in an emergency, but I can't find and trigger the SOS feature!? Again, thanks to the Twilio messaging and phone call services, with the help of our image and sensor data, now any blind person can Quickly intimate the authorities of the emergency along with their GPS location. (This includes auto-detection of hazards too)
EZ Shoppe - It's not an easy job for a blind person to access ATMs or perform monetary transactions independently. And so, taking this into consideration, with the help of superbly designed Capital One Hackathon API, we have created a server-based blockchain transaction system which adds ease to your shopping without being worried about anything. Currently, the server integrated module supports customer addition, account addition, person to person transactions, merchant transactions, balance check and info, withdrawals and secure payment to vendors. No need of worrying about individual items, just with one QR scan, your entire shopping list is generated along with the vendor information and the total billing amount.
What's up Doc - Monitoring heart pulse rate and using online datasets, we devised a machine learning algorithm and classified labels which tells about the person's health. These labels include: "Athletic", "Excellent", 'Good", "Above Average", "Average", "Below Average" and "Poor". The function takes age, heart rate, and gender as an argument and performs the computation to provide you with the best current condition of your heart pulse rate.
*All features above can be triggered from Phone via voice, Alexa echo dot and even the wearable itself. **Output information is relayed via headphones and Alexa.
How we built it
Retrofit Devices (NodeMCU) fit behind switchboards and allow them to be controlled remotely.
The RSSI guidance uses Wi-Fi signal intensity to triangulate its position. Ultrasonic sensor and camera detects obstacles (OpenCV) and runs Left and Right haptic motors according to closeness to device and position of the obstacle.
We used dlib computer vision library to record and extract features to perform facial recognition.
Microsoft Azure Cloud services takes a series of images to describe the activity taking place.
We used Optical Character Recognition (Google Cloud) for Text To Speech Output.
We used Google Cloud Vision which classifies and locates the object.
Twilio API sends the alert using GPS from Phone when a hazard is detected by the Google Cloud Vision API.
QR Scanner scans the QR Code and uses Capital One API to make secure and fast transactions in a BlockChain Network.
Pulse Sensor data is taken and sent to the server where it is analysed using ML models from AWS SageMaker to make the health predictions.
Challenges we ran into
Making individual modules was a bit easier but integrating them all together into one hardware (Raspberry Pi) and getting them to work was something really challenging to us.
Accomplishments that we're proud of
The number of features we successfully integrated to prototype level.
What we learned
We learned to trust in ourselves and our teammates and that when we do that there's nothing we can't accomplish.
What's next for The Third Eye
Adding a personal assistant to up the game and so much more. Every person has potential they deserve to unleash; we pledge to level the playfield by taking this initiative forward and strongly urge you to help us in this undertaking.