Imagine yourself walking past a poster board, or sitting at a presentation. You see a poster about a really fascinating event, and you take a picture of it. One week later, however, you missed the event because your phone gallery does not remind you of the event as your calendar.
What it does
We are introducing ---- Event Scanner! Event Scanner is an ios app that recognizes the date & location information in a picture you take, and automatically adds that event to your calendar (with your permission) so that you will never forget about any interesting event!
How we built it
We built the app using Swift on Xcode. We implemented Google´s OCR (Optical Characters Recognition) API to detect and extract the text information from a picture that the user can either take or pull from the gallery. In addition, we integrated Natural Language Processing (NLP) algorithms to process the text extracted and looks for the date & time information within the text. Finally, using Apple's EventKit, the app can add the event to user's calendar with permission granted.
Challenges we ran into
For the date time and location extraction, we had challenge in building the model. Originally we thought of using a ML model for location tagging. Due to the limited data, we ended up building the algorithm merely targeted for Rice building search. If we can get back more data in the future, we can train a ML model to make the location tag more accurate.
Accomplishments that we're proud of
We were able to use natural language processing to create a smart app that understands date, time, and location presented in a variety of formats. We transferred large files via URL session run in the background thread, so that the main thread of the app would remain uninterrupted while we waited for image analysis results from Google Cloud.
What we learned
Three of our team members do not have prior ios development experience, and we gain such an important skill in 36 hours. We are now much more familiar of Google API's and Xcode's libraries.
What's next for Event-Scanner
Event-Scanner will perform better with a larger training data set. We plan on making larger and more precise data sets to train. We look forward to also implementing sdks in our app for more complete functions. Last but not least, development on Android as well!