Inspiration
In our increasingly chaotic lives, it’s hard to keep track of our thoughts and mental health. The seemingly insurmountable pile of thoughts and tasks in our brains lead to stress, negative self image, and low self confidence. Journaling is one of the best ways to organize your thoughts, clear your mind and store your memories. Keeping in theme with HackGT X's "Journal of Memories", we came up with MoodMosaic; a visual approach to journaling!
What it does
MoodMosaic is a web application that uses inputs given by the user to generate an image depicting the mood and tone of the entry. This image can be interpreted by the user to show the tone of language used to describe themselves or events in their day. An overly negative tone would result in a gloomy looking mosaic, and a positive tone would result in a cheery one.
How we built it
Our application takes a text input or simply speech input from the user’s end and integrates with OpenAI’s Sentiment Analysis Natural Language API to generate a sentiment analysis and sentiment score from the journal entry. This output is then integrated with OpenAI’s Image Generation API using the Dall-E model as a prompt to generate an apt image for the sentiment. This image, or mosaic, is then associated with the respective diary entry and depicts the user’s mood in the whole entry.
Challenges we ran into
Initially, we had difficulty integrating the Image Generation API endpoints into our product, since there are very few Dall-E based Image Generation models that are accessible. We experimented with a few before settling on OpenAI’s Image Generation API endpoint. Even this API Endpoint is not completely open-access, therefore we would like to test out different image generation models and APIs to find something that improves accessibility.
Accomplishments that we're proud of
We are really proud to have identified this need which has such a wide reach. People all over the world struggle with stress and as a result tend to neglect their mental health. A task as simple as journaling makes a world of difference and it is very important for us to deliver an immersive and exciting experience to users that also gives them an insight into their own thought process and the tone of the language they use to describe their day to day experiences.
What we learned
We learned that simple prompting of image generation models with sentiment messages didn’t always lead to the results we wanted! We had to use a mix of sentiment score and sentiment analysis text to prompt the image generation model to receive viable images to convert into mood mosaics.
What's next for MoodMosaic
We have a lot of exciting ideas in mind to expand on our product! One of these is a feature where if the detected tone of the journal entry is overly negative, the application will bring up a past entry that is positive or showcases an achievement of the user, to lift their spirits, remind them of their capability and boost their self-worth. We would also like to experiment with different image generation models to come up with more accurate depictions of text sentiment via images.
Log in or sign up for Devpost to join the conversation.