What the F--- is that?

Search of Pringles
Search of Microsoft
Searching for the word Microsoft
Searching for the word Pringles
Search of Pea
Searching for the word Pea

Inspiration

Every restaurant ever has the most concise and minimalistic menus. We always found ourselves having to manually research the food and drinks provided, as the only thing we see is text. We wanted a way to automate this process to make it more friendly and less bloated, and we believed we could provide clean user friendly solution that provides ultimately what you need to make your decision on what to eat.

What It Does

What the F--- is That? takes a camera input source focused on text and on a simple tap, an image is captured. Optical Character Recognition is applied on the image and is then parsed into a readable format. What happens next is that it will analyze the text, image search of the product and provide nutritional information of the various food and beverages.

How We Built It

It uses a camera on a device to take a picture of the text, and then we applied an Optical Character Recognition API on it and parse it into a string, and the Bing Image Search API to searches for results in parallel with an API that retrieves nutritional info on the scanned item, which is then displayed in a sleek user interface.

Challenges We Ran Into

Optical Character Recognition is insanely tough to implement, so finding an API that did it proficiently was a challenge. We tried multiple API's but ultimately we decided on the one that suited our needs.

Accomplishments That We're Proud Of

We are proud not only of the motivation to create this product, but how quickly we were able to implement core functionality, within a single weekend.

What We Learned

We realized that we have the potential to make and expand our product into the wider market, and take our innovation to the next level.

The Future

Our goals towards the future is to use this in the wider market. We realized we made something that is extensible and dynamic enough so that we would be able to analyze things in pseudo-augmented reality wherein we would be able to live display information beside it in real-time. At the Hackathon we realized that we would be to extend our software to live Optical Character Recognition tracking within Augmented Reality environments such as Microsoft Hololens, which could be used for simple user functionality, but as a new innovative way to express ad space.