The Price is Right

Using computer vision to identify products in real time -- guess the price of things you interact with every day!

Comment

Inspiration

Like Trivia HQ did for trivia game shows, we're bringing price-guessing game shows to the everyday user.

What it does

The Price is Right is a novel, social-based augmented reality mobile platform and game to establish an interface between the real world and the every day needs of consumers. Simply take a picture of any sort of item, and using eBay's advanced convolutional neural network-based image search, we will scour the web for a close fit. Then, put on your thinking cap, and guess the price of the closest product found. Share your results with others, views the daily challenge, and see other people's relevant, funny, and interesting photos! Although centered around a game, this application has serious potential to drive people to easily find and purchase items in a fun way, and establish large data sets to further machine learning models.

How we built it

First, we developed a mobile application using Android Studio, utilizing many different Views, Activities, and Styles to design an elegant yet simple user interface and handle basic logic. Next, we used Google's Camera API to allow the user to take a photo of objects which were then turned into Base64 Strings to be later used in the application. Our core product rests on our novel use of eBay's Browse API; we used this interface to search for the photos that the user took, matching them to the closest product sold on eBay, which we in turn pulled the data for. Finally, we allowed the user to guess and compare the price of the object, and used Google Cloud's real time Firebase to store the photos and results in a custom backend database. This then allowed us to do things such as show the most recent product lookups and price guesses of other users, allow users to upvote searches and guesses they like, and to store past user searches (perhaps to construct a tailored advertisement profile).

Challenges we ran into

One challenge was writing a function called bitmapToBase64 that allowed us to convert a bitmap to a base 64 string that was necessary for using the browse eBay API. This involved compressing the bitmap and using a byte array output stream to encode the bitmap in base 64. Another challenge was using eBay's APIs to match photos to similar products and return relevant information. One of our greatest challenges was integrating disparate tools like Firebase, eBay APIs, and Picasso into a seamless, cohesive app that uses their strengths effectively.

Accomplishments that we're proud of

We designed a cohesive, visually striking app that successively incorporates a number of challenging components. The Price is Right integrates camera features, eBay's Browse API, Firebase, and computer vision into a mobile app.

What we learned

We learned how to use Firebase and eBay APIs in Android app development, designing a nontrivial (mobile) UI as a group, and maintaining version control with multiple people working on a project at once. We also learned about image processing and computer vision.

What's next for The Price is Right

The Price is Right is more than just a game; it is a social platform and a treasure trove of data. Next steps include allowing users to predict more than just price - for example, allowing them to predict star ratings on a product to sniff out products with potential fake reviews. Furthermore, we plan on incentivizing users to play by offering small discounts/rewards based on the products they search and links they click on. The data gathered is invaluable in refining pricing models (seeing how a bling market prices products), as well as further refining computer vision models such as eBay's image search (by asking how the user how well the image search performed and using the value as a heuristic). On the game side, The Price is Right gameshow has over a a hundred different variations of mini-games representing different pricing challenges that we could implement. As users play more and more, we can make specialized advertisement profiles based on what they search for, allowing sellers to reach a more focused market. Finally, we would add more social features, such as a public leaderboard, sharing correct guess streaks with other users.

Built With

Share this project:

Updates