Smart Ads which has two sides - customer benefits and enterprise profits
Smart Ads Flow diagram
Login Page on Android App
Click Camera on Android App to Scan bills.
Upload Bill Image for OCR analysis on AWS
From office to Home at 5:30pm , the user is provided with Tuna offer enroute
SMS sent via Twilio app to get the tuna offer
Who has not installed AdBlock extensions? Ads are synonymous to annoyance and irrelevance. Most companies' sole income is through advertisements for example blogs, google news. Ads are a big deal for the survival of many companies and at the same time if right Ads are sent to right people at the right time they can be a real benefit to customers.
We propose a solution which:
- Provides useful ads to users without annoying them.
- Proposes very less or minimal detour from their regular activities.
- Enterprise - Providing customized offers to keep loyal customers happy.
Smart Ads are crucial for customers - enterprise ecosystem.
What it does
Presenting Smart Ads - which provides advertisement to the customers based on
- Bills OCR Analysis - User scans and uploads bills via phone app and analysis is done on it to gain insights into user preferences for potential offers.
- Google Calendar Sync - Uses Calendar events to decide ad notifications. For ex: If the user is going to a birthday party for the weekend, ads related to offers on wine or gifts can be shown on the route.
- Location based: Uses the current location of the user and calendar event to notify ads to users.
How we built it
- Android app - Used for scan and upload of purchase recipts, and periodically sends location to our cloud endpoint.
- AWS Server - All OCR analysis done here, from getting text from scanned image, to categorizing purchases, and learning user preferences. Calendar events for the user are synced to get upcoming event details. REST endpoint is provided using Flask, to get potential offers from partner enterprises based on location. Matched result offers are pushed by SMS to user via Twilio.
- JS Application - Hosted on AWS to provide Map Visualization to demonstrate the backend application features.
- ESRI - Mapping Location Name to (lat,long) co-ordinates (geo-coding).
- Amazon SQS: To queue the location data and process at regular intervals.
- TWILIO - Sending SMS to user for pushing offers.
- PYTESSER - Tesseract OCR Python lib.
- OpenCV - Crop and perspective transformation for image manipulation.
- GOOGLE MAPS - Google Maps to visualize the JS Application.
Team Members and Responsibilities
- Aniket Dhuri - Android App and Flask REST API
- Chethan Nannivala Thipperudrappa - Flask REST API and JS Application
- Saurabh Sakpal - ESRI API , JS Application and Calendar
- Utkarsh Verma - OCR , Image Transformation and JS Application
Challenges we ran into
- Android application crash due to OS terminating location listenerService indeterminately while running in background- resolved using startForeground().
- REST API providing blank response because Amazon AWS was not responding due to large number of requests from HackNC users. It was resolved by connecting to internet using cellular network.
Accomplishments that I'm proud of
- Worked on various platforms to have complete exposure to the frameworks and technologies used.
- Choosing team members who were motivated and provided constructive feedback on the decisions taken during the 24 hours of coding.
What we learned
- Perseverance - bugs everywhere, don't give up - fix'em all
- Android - quick refresher for me after 4 years.
- Flask - REST endpoints using Flask.
- Fun - Enjoy while you code , especially pair programming.
- OCR - Using tesseract OCR.
What's next for Smart Ads
We were discussing about how to leverage the use case for benefiting the enterprise, and came up with various solutions ranging from -
- Adding support in Android to add offers based on keywords and/or
interfacing with restaurant APIs to provide customized offers to each individual.
- Providing a feature to users to add list of the items they purchase frequently ( based on points the user has accumulated by buying more offers ).