Eyesight

Inspiration:

After conducting research we found that worldwide, an estimated 2.2 billion people have a near or distance vision impairment, which indeed is a staggering amount of people. While we may not realize it, people who are visually impaired often have difficulty to identify objects and navigate the world independently. This results in them having to become dependent on someone and others to help them. Eyesight changes that.

What it does:

Introducing Eyesight, the beginning of the next generation of visual accessibility technology. Eyesight is a mobile application that uses the power of artificial intelligence and integration with remarkable APIs. Eyesight aims to improve the lives of people around the world by providing them with the tools they need to increase their independence. This is just the beginning of what Eyesight and other accessibility solutions can do for society.

How we built it

Eyesight is powered by one of the world’s most powerful computer vision algorithms through Microsoft Azure and built on a cross-platform application development framework with React Native, providing the user with accurate yet quick descriptions.

Challenges we ran into

We worked day and night to turn this app idea into a working reality. During development, we encountered many issues and challenges from issues with sending data to the Azure function to managing state within our React Native project. However, I would like to discuss two challenges our team had that played an important role in the scalability of our app, which were handling large volumes of API requests and providing a seamless cross platform experience for our users. Every time the user opens one of the two modes, API requests are immediately sent to one of our three Azure functions. Typically, this is not a problem if a single user is using the application over a period of a couple minutes However, when either the amount of concurrent users increases or when a user uses the app for a long period of time, it has the potential to either significantly increase the cost of operating the application or completely shut down the services due to maxing out a plan’s API calls. For example, during testing, we realized that we would quickly run out of API calls for the sightengine API which would have prevented object detection on the application. We knew that we had to limit API calls from the application so we decided to implement several features into that app that would do just that. The first change we implemented was motion controlled API requests for object detection. Instead of continuously sending API requests, we created a condition that checked the device’s accelerometer and gyroscope for significant movement or no movement at all. In either case, the application would not send an API request. We also implemented a JavaScript timer using the setInterval method. This limited how often API requests could be sent to one potential request every 2 seconds. Both of these features significantly reduced the amount of API requests sent to Azure, reducing operating costs and increasing the speed and reliability of the application for the user. In addition to limiting API calls, we also had a challenge in developing a mobile application for both Android and iOS in only four weeks. We looked at multiple different solutions, from building native applications for iOS and Android to building a webapp. Although good ideas, they would either take too much time to create or would severely limit the potential of the project. Then we came across libraries such as React Native and Flutter. Both React Native and Flutter are frameworks for building cross-platform apps. This would allow our team to use a single codebase to develop for both the iOS and Android application, cutting development time significantly while keeping the experience native. Our team believed that either one of these frameworks would work perfectly for our project and because we had some experience with React and React Native, we decided to use this framework for our project.

What we learned

Throughout the 4 weeks of extensive app development, we learned to harness the power of Microsoft Azure serverless technology and integrate it with React Native, a framework for developing cross-platform mobile applications.

What’s next for Eyesight

As we continue the development of Eyesight, our goal is to increase the ease of use of the application, increase the responsiveness of the application, and pave the way for new accessibility technology. To increase the useability of the application, we are planning a series of changes to both the application and to our Azure functions. For example, we are planning a full overhaul of our user interface to comply with the Web Content Accessibility Standard. In the application responsiveness, we would like to reduce the number of APIs we use to decrease response time when using the application. We would also like to integrate compression algorithms to future decrease response times for certain processes. Finally, Eyesight will remain an open source project so that people around the world can both use and expand the possibilities of Eyesight.