EMOXT

EMOXT Logo

Inspiration

During the COVID-19 pandemic, online texting has replaced in-person interaction. To solve this problem, we wanted to inject emotion into text - through emojis! We were very interested in using Assembly.ai in our project, which transcribes audio to text using machine learning. We decided that we wanted to then attempt to add appropriate emojis to the transcribed text using machine learning. This adds emotion to the text, something that gets lost in the translation from audio to text.

What it does

EMOXT takes audio recorded from your device's microphone, and transcribes it into text. Then, appropriate emojis are inserted into the transcribed text based on machine learning techniques that predict the emotions needed.

How we built it

We used Assembly.ai to do the voice to text transcription, and machine learning using scraped tweets from Twitter as a training dataset. We used flask to connect the front and back ends.

Challenges we ran into

We had a difficult time converting the recorded audio into an mp3 file that we could then use in Assembly.ai to transcribe. After a long time searching, however, we managed to find a solution!

We had trouble finding datasets and converting them into proper formats for machine learning to proceed. But we managed to solve that using Python operations on .csv files.

Accomplishments that we're proud of

We are very proud of being able to use and implement machine learning algorithms in such a short time span, and that we put forward a finished project.

What we learned

All of us learned a great deal while completing this project - from using Assembly.ai for the first time, to scraping tweets, to connecting the frontend and backend for a finished project.

What's next for EMOXT

In the future, we'd love to expand on our initial idea. We could collect a larger dataset by crawling from social media, and add more emojis for when the dataset becomes larger!