Yupon AI

An example of an ad graphic generated with DALL-E. Current version now includes a description/promotion.
Mobile app customer side where customers can look for restaurants to order from and get rewards
Customers can also get points or use points to purchase food
Create campaigns
Restaurants can upload their menu and automatically generate rewards app
landing page for businesses

Inspiration

Customer Interviews

-We have interviewed multiple small Restaurants: all of them have highlighted the the same pain points

Eat Bubbles (Boba Tea)
Tropical Island Cafe (Jamaican Food)
Masa (Japanese Food)

-Our project allows small businesses (just restaurants for now) to engage in online advertising at a similar level of sophistication as large-budget corporations. Running online ads for your restaurant NO LONGER requires:

Photography Skills
Photo Editing Skills
Graphic Design Skills
Familiarity with Advertising Interfaces and Targeting
The ability to code (or set up) a mobile ordering platform

What it does

Yupon AI is an AI marketing platform for locally-owned restaurants.

It has 2 distinct features:

Upload a menu to automatically generate a mobile rewards points app. Restaurants can easily create a mobile ordering/rewards app just by uploading a picture of their menu. Yupon will use Google Cloud Vision's optical character recognition machine learning tool to automatically detect the menu items and generate a loyalty app based on that.

The app allows users to order with money or points and automatically generates the points value system. The app also has search, profile, and history capabilities.

Generate realistic AI food graphics to automatically start running ads and detect which graphics perform the best, allocating more funds towards those graphics. Restaurants can easily create Facebook ad campaigns by just providing a budget, promotion, and description of the food. Yupon will use the description to automatically generate a few professional pictures of the food. After that, Yupon integrates with the Meta Marketing/Ads API to start running ads that point to the rewards app. Using machine learning, Yupon can optimize to see which ad performs the best and allocate more funds towards those.

Furthermore, Yupon can detect which placements perform the best (Stories, Reels, Feeds, Messenger), as well as using data on age, demographics, and interests to further optimize ad campaigns.

How we built it

Our team coded for about 15 straight hours with no stopping except for food. George worked on the Business web dashboard for automatically creating and storing the app menu details through Google Cloud Vision and automatically generating and optimizing ads through DALL-E and Meta Ads API. Jerry worked on the mobile app rewards points system and UI. Michael helped pair program and read documentation.

Tech Stack

Web Dashboard for Businesses

-Next.js 13

-Google Cloud Vision OCR

-OpenAI DALL-E API

-Meta/Facebook Ads API

-Auth0

-MongoDB

-Google Cloud Storage

-Tailwind CSS

Mobile App

-Reactive Native/Expo

-Auth0

-React-Redux

Challenges we ran into

The documentation for the Meta Marketing/Ads API is incredibly outdated. The current version is 18, and the documentation still contains references to verison 2. The documentation did NOT mention some REQUIRED fields, so the only way to figure out what was going wrong was to keep testing requests through the Graph API explorer and reading error messages until it worked.

Jerry had never used React Native before, but since he was familiar with React, it was easy for him to pick up.

Accomplishments that we're proud of

Despite the challenges, we were able to sucessfully complete 3 complex AI/ML API integrations (Google Cloud Vision, OpenAI DALL-E, Meta Ads API) while still making the user mobile app side in time with a great UI.

Our integrations qualify us for the following prize categories:

Google Cloud

Auth0

Overall