Blinq

Inspiration

In 2015, there were an estimated 253 million people with visual impairment worldwide. Of these, 36 million were blind and a further 217 million had moderate to severe visual impairment. In context of Singapore- the problem worsens as the risk of blindness increases fifteen-fold for aged 50 to 80 and above. The visually impaired face problems in all aspects of life including travel and transport. As Singapore moves towards establishing itself as a SMART Nation , it becomes absolutely imperative to provide practical solutions to those in need.

As concerned citizens who have witnessed these problems in our own family, we felt that it was our duty to work towards building a safer, inclusive society.

What it does

Blinq incorporates a SMART cane along with a technological assistant application which utilizes robotics, machine learning and IoT to provide a one stop solution to the daily struggles of a person suffering from visual impairment.

SMART Cane Obstacle Detection Our SMART Cane is equipped with Ultrasonic sensor (HC-SRO4) attached to the arduino and linked to mobile application via bluetooth (HC05) that detects any incoming obstacle in a 2 meter radius. This enables the user to be aware of any possible obstruction in his/her immediate surroundings while walking. Our SMART cane is also equipped with temperature and humidity sensor which warns the user of incoming temperature changes and rain.

ML Live Environment Tracker There are often instances when a visually impaired person might not have any idea of their surroundings and might need assistance in identifying nearby objects. Therefore our ML Live Environment Tracker enables the user to point the camera around him/her and the program will list down all objects in vicinty such as Cars, Trees, Pedestrians , Electronics , Stationery, Chairs etc through spoken command.

Tesseract Text Reader We understand how it might be extremely frustrating for a visually impaired person to not be able to read notices / menus / food labels and hence we have developed a function that allows the user to point their phone camera at the object and ask the application to identify and read the text for them through spoken command.

Implementation of Code

Modified Yolov2 ML object detection model using Tensorflow and Keras. Exported model and labels to TFLite format for integration with Flutter. Connects with the mobile phone camera for live detection.

Implemented Text Recognizer model for reading text from images and deployed it on Flutter. Connects with the mobile phone camera to snap picture for recognition.

Connected HC-SR04 Ultrasonic sensor, and DHT 11 temperature and humidity sensors to Arduino microcontroller.

Programmed the Arduino microcontroller to connect to the mobile application using the HC-05 Bluetooth module.

Executed speech recognition model so that the application can recognize set of spoken instruction commands by the user.

Recorded data in text format on the application is converted to speech and spoken out by the application.

Challenges we ran into

Incorporating so many functionalities that were coded across Arduino, Python and Ruby into one common flutter platform was extremely challenging considering the time crunch. We had to train our huge ML Model, build our final hardware product, and work with faulty parts.

Accomplishments that we're proud of

We are proud to have successfully developed an entire app and a hardware product that can actually make a difference in this world in a very short duration of time. We are proud that we were able to implement our theory knowledge into practice and also work across a wide variety of platforms such as Python, Arduino, C, and Ruby.

What we learned

Each member of our team had different strengths when we started working - some of us excelled in Machine Learning, while some of us were fluent in App Development and IOT. At the end, while working with each other - we were able to transfer our knowledge and learn from each other. We feel we now possess a deeper understanding of this vast discipline.

What's next for Blinq

●‘Fall Alert’ feature that uses accelerometer to detect when the user falls down, and automatically calls user’s emergency contact. ●Community feature that matches volunteers to users that might require help in certain situations such as travelling to the doctor and urgent assistance. ●Customizable component that can be used to fit the sensorboard device onto any cane efficiently. An option to have a smart shoe as an alternative for users that do not wish to use a cane. Improving the structure and design of the sensorboard for greater marketability.