Melissa AR

Us at the Blanket Fort event
Ethan trying his ARCore code

Inspiration

We wanted something feasible and fun; @YilinW28 came up with the idea of an Instagram filter because of how popular they're right now. We later found out that the famous 'Which Disney Character Are You' filter assigns the characters randomly, so we thought of making the selection with some kind of actual criteria.

What it does

It uses Microsoft Azure to identify key features of the user's face and then compare those features with a group of images. Next, it returns an index value of how close the face is to the group of selected images and then it shows which animated character resembles the most to the user. The result gets displayed as a picture of the character on the user's forehead with the use of augmented reality software, just how it would in the original filter.

How we built it

The heart of the system is the Face API from Microsoft Azure. The idea is to take a snapshot of the face while the app shows you a live feed fo the frontal camera. Then use a modified version of the Find Similar code to compare that snapshot with a group of photos and select the one with the highest matching index. Since the Face API has trouble with identifying features from animated characters, we got inspired by Chloe Condon and wanted to try using pictures of Disney cosplayers to make the comparisons and then assign the animated character depending on how similar the cosplayer's features were to the user.

We employed the ARCore android library to place objects on the user's face during the live feed, and the idea was to put a frame on top of the user's head and have the images inside the frame flick around until the comparison was done and then it would show the result. Very similar to the original 'Which Disney Character Are You' filter.

For the UX design, we employed ProtoPie. Since the theme was Disney characters, we wanted to make it look colorful and playful. The process started by looking at different Snapchat filters and deciding for a color palette that was appealing and simple.

Challenges we ran into

We started with little to no experience on app development and tried our best to do as much as we could. Even though giving the first steps seemed daunting, we just tried to have fun and to take it easy. This allowed us to skip some frustrations and make for a good team environment.

We had problems implementing the java code because of our inexperience and several software compatibility issues, some of which have been solved through research and patience.

Accomplishments that we're proud of

We met at the team mixer that was organized at the opening ceremony and an hour later we already had an idea that we all liked. Coming up with something is hard, and your mind might be blank when you're trying to force it to be creative, but give it time, speak up even if it sounds dumb because as a team you could find a way to make it better, and research to get inspired by what other people have done with similar ideas.

We got some functionality working and went from 0 to acceptable when it comes to app development. We learned to use new tools and feel good about what we got considering our experience and the time we had.

We got a good team dynamic. We communicated throughout the whole process and set up team updates every hour to see where everyone was at that moment.

What we learned

Making an android app from scratch in a short time is not an easy task. Resources are plenty but the learning curve is steep. If you're participating with a competitive mindset, preparation is key.

We feel that one of the most valuable things you learn at a hackathon is how to work with people and establish an effective dynamic in an unknown environment. It's a test of character. Knowing how to deal with human relationships is a skill that everybody needs.