QWERSTAR

Camera screen for detecting different user responses
Quote page for feedback

Inspiration

We wanted a fun little feel-good project that would be both entertaining and helpful — an app that can read your mood and deliver a cute pick-me-up as well as empowering queer facts!

What it does

Using machine learning technology, it detects whether the user is showing a thumbs-down or thumbs-up, and based on that input, chooses an appropriate response from the array of responses available.

How we built it

We used the Google Teachable Machine for creating a model that would accurately determine the difference between a thumbs-up and a thumbs-down, and exported that model as tensorflow-lite and linked it to an iOS application.

Challenges we ran into

Converting the model to the tensorflow-lite and using that model in a Swift environment was a challenging aspect, as the data had to be parsed using regex into appropriate forms. Another difficulty was training an appropriate model using the right amount of data that would give us consistent results. Finally, our first and foremost challenge was obtaining a good and publicly available dataset, since in the beginning we intended to use smiles and frowns as our indication of mood. When this proved near-impossible, we decided to use hand gestures instead.

Accomplishments that we're proud of

The accurate model that can almost always differentiate between a thumbs-up and thumbs-down, an iOS application that can appropriately use the model in hand, and the calming user-interface.

What we learned

How to use a Google Teachable Machine, a baseline knowledge of how they operate and what they need, and parsing/transforming data.

What's next for QWERSTAR

With a large enough dataset, we could put our original idea into motion, detecting the difference between a smile and a frown, and making QWERSTAR work on a more subtle level for all users.