During our first night at HackGT, we began by exploring the different floors of both the Klaus Research Building and Classroom Building. One challenge that caught our attention was the PDI challenge to use Google MobileVision to interpret a gas station sign - one of the worst images to feed to that particular API. As a group of people with minimal tech experience but a shared love of challenges, we decided to participate in the PDI challenge.
What it does:
A user with the iOS app can take a picture of a gas station sign, which will be then run through the Google MobileVision API. The API returns a series of words and numbers that it detects in the image, and then we process that data to sort out what data is actually gas prices. After we obtain the data, we drop a "pin" on the map, which can then be clicked on by other users and alerts them of the gas prices at that particular station.
How we built it:
We used XCode, Apple's iOS app-building environment, to put together the app, which was coded in Swift. The Google MobileVision API is accessed through Firebase, which is an app development platform managed by Google.
Challenges we ran into:
Throughout the hackathon, we ran into a multitude of problems, both technical and strategic. Our access to MobileVision (and the logic that detects gas prices from the data extracted from the API) was originally coded in python. Because of this, we had to decide whether to attempt to access our python code in XCode or to just rewrite the MobileVision access entirely in Swift. We eventually decided to translate both the API access and data interpreting logic into Swift. One PDI Challenge specific issue that we ran into was that they weren't lying - Google Vision absolutely hates gas station signs. The problems that we encountered ranged from the API detecting only the first digit of the denominator of a fraction to it ignoring chunks of the sign entirely. This is due to the wide range of font sizes, colors, and fonts that can be used on one sign. Because of this, we are still working out some issues and will continue to change our interpretation logic as we run into other problems.
Accomplishments that we're proud of:
We are proud of the fact that, by and large, we were able to identify issues with individual gas stations and generalize solutions for them that can work with a multitude of sign styles and sizes. Most of all, we are proud of the amount that we have learned about Swift and iOS app development throughout the hackathon, which is nothing something that a majority of the group was familiar with previously.
What we learned:
The main thing that we learned throughout this hackathon was how to work together as a team on a singular project and how to divide work between multiple people. This experience has helped us to get a clearer glimpse into what working on a team for a long term project during an internship or job might be.
What's next for PDI Challenge - GasStationVisionMapped
Our next step in project development is to make our app cross-platform. We want to learn how to use Android Studio to program an Android app similar to our in-progress iOS app. We also want to edit the map locations so that when a "pin" is dropped at a gas station, it will automatically disappear after 24 hours, prompting users to continue providing real-time data for the most accurate information.