View information about locations around you using augmented reality. Text appears as if it exists in 3D Space in the place's location.
Get relevant information about an object just by pointing your camera at it.
InformAR has the option to read out what it sees to help give vision back to the visually impaired.
Augmented Reality allows you to instantly view information about locations around you.
"International tourist arrivals up 4% reach a record 1.2 billion in 2015" - World Tourism Organization
Last year, the amount of tourism worldwide jumped 4% - thats the most this decade. In addition to this, we were immediately inspired the first time we saw working augmented reality in an iOS app, and believe that it is, along with virtual reality, a technology of the future. We wanted to utilize it in our app.
What it does
InformAR uses augmented reality to display information about various locations in your area. This means that you can point your camera at any building near you and instantly get information about it, superimposed in 3D space on the building. Simply tap on any location you see to get an Uber ride there.
In addition to this, InformAR can recognize objects and instantly display information about them. This means that you can point your camera at any object and tap on it to instantly gain information about it.*
Finally, we realized that object recognition had the ability to help the visually impaired, so InformAR can read out what it sees to help tell people who are visually impaired what's around them.
Tourism Industry "International tourist arrivals up 4% reach a record 1.2 billion in 2015" - World Tourism Organization The world tourism industry is booming. Why spend money on a tour guide when InformAR allows you to instantly get the Wikipedia article about a building by pointing your camera at it? Instantly view information about unfamiliar buildings in augmented reality.
Foreign Language Speakers "The number of foreign-language speakers increased 2.2 million between 2010 and 2013." - cis.org "One in five US residents speaks a foreign language" - cis.org Don't know what an object is called? Simply point your camera at it and find out.
The Visually Impaired InformAR helps give back vision to the visually impaired. A person who is visually impaired can keep the camera in their shirt pocket and simply tap the phone to get a readout of what is in front of them.
Curious people InformAR allows naturally curious people to instantly find information about objects around them that they see in everyday life. For example, you can now find out about Coca-Cola's history by pointing your camera at a Coke Can.
How we built it
We built the augmented reality framework ourselves with math that we found online. We measured the tilt and gyro data from the iPhone, in addition to GPS location and Compass direction to give the appearance that text existed in 3D space. We got the information that we displayed from Google Places.
We used ClarifAI's Artificial Image Recognition API to recognize objects that the user points their camera at, and then we scraped Google and Wikipedia to return a relevant 550 word summary of the object and display this to the user.
In addition to this, we used core iOS technologies to say the information to help the visually impaired.
Challenges we ran into
- At first, we tried to implement an existing Augmented Reality framework but we had lots of problems getting it to work because it was written in Objective-C in 2005, and when we got it to work it did not perform as expected. So, we decided to build it ourselves.
- Webscraping Wikipedia. Since about every other word in Wikipedia has a link, we had a little bit of trouble getting the information.
- Getting Augmented Reality to work on Web.
Accomplishments that we're proud of
- Working Augmented Reality on iOS that we coded ourselves.
- Working object recognition
- Working iOS Application!!!!
What we learned
We spent a lot of time learning the math behind Augmented Reality.
What's next for InformAR
- Adding support for Android.
- Displaying more information for places, such as reviews and ratings.
- Adding multiple language support to help english language speakers in other countries.