Pop2See

• 

  Ηome Screen

• 

  Identifying Objects

• 

• 

• 

• 

• 

  Νavigation - 1

• 

  Navigation - 2

Inspiration

We got inspired by a visit to a hospital with a blind friend, who couldn't do most things by herself. We helped her with her medicine, insurance and hospital visit.

All actions were not accessible at all, so we decided to experiment with a solution for this problem.

There are 250,000,000 blind or low vision around the world (official OpenAI announcement).

Furthermore, blind people use their phones as their main everyday tool (as sighted people do), only that they get all information over audio, instead of the screen.

In addition a very interesting fact is that if you ask a blind person, what are their most important problems, their answers usually include accessing information and navigating within spaces.

What it does

It helps blind people use their phone's camera to access information and soon navigate within spaces.

Accessing information is addressed by taking a photo with their camera, and then having AI to describe, having built specific use cases. For example, identifying a product at the supermarket, or a medicine at home or in the pharmacy, learning the expiration date of package, or understanding where the door of a building is.

As our blind testers say, "Is this yogurt 2% or 5% fat?", "Is my bag on that table?", and "how many apples do I have?"

How we built it

We have built our MVP by combining low code technologies with AI. More specifically, we are using FlutterFlow to build our native apps, Google Firebase as the database, and Google Gemini as an AI model.

Challenges we ran into

The most challenging aspects of our project were:

• identifying the use cases that are actually useful
• integrating AI in our workflow
• making AI work as we want

Accomplishments that we're proud of

We are very proud for the reaction of our first blind tester, who got really excited during the test. She was amazed by the details she could get, and even found a bag in her place that she didn't know where it was.

What we learned

We learnt that Low Code + AI can do miracles, especially when it comes to social impact.

We also learnt how to integrate AI models in Low Code implementations and explore Google Console and Gemini in depth.

We are looking forward to the next steps of this app.

What's next for Pop2See

Our next steps include:

• building more use cases to address real needs of blind users
• working with companies to create in-app experiences for their products e.g. recognizing Nespresso capsules (decaf vs caf)
• testing the app extensively with blind users

PS. If you want to try out our apps, we need to give you credentials and renew our API authentication keys

Built With

• firebase
• flutterflow
• gemini