Image taken and automatically described with SocialEyes
What it does
Allows users with visual impairments to share photos on social media (Twitter) and with friends through MMS. The user is able to take a picture and then the app will read the content of the image back to the user. The user can then decide to share the image to Twitter, send it as an MMS or simply take a new photo to get to know the environment.
How we built it
The application is using Microsoft cognitive services to figure out what the picture is showing. The Twilio API is used to send text messages. The application is created as a web-application using Flask. Jquery libraries are used to allow for touch interface gestures.
Challenges we ran into
The HTML5 camera module was far from as mature as we had hoped for. During the night we started porting the app to Android. It was though limited how far we got with this due to our limited knowledge in the area.
Accomplishments that we're proud of
We actually build a working prototype that we think can be of good usage.
What we learned
How to work with HTML5 media objects. How to create an Android application.
What's next for SocialEyes
- Implement speech to text to send SMS and MMS to friends and family.
- Add user information from selfie picture
- Voice interaction navigation
- Make it faster!!!
Info: The thumbnail and the image are created and sent to twitter using the SocialEyes app.
domain.com doesn't show our domain. But the IP should work ...