MoodScribe - Real-time AI Social Sidekick

Landing Page
Log In Page
Conversation Page
All user's profile picture change based on the tone of the conversation after clicking "generate" to analyze conversation

Inspiration

MoodScribe was born from a simple yet powerful idea: to create a tool that enhances the way people communicate online. We were inspired by the power of words to connect, uplift, and convey empathy—even through a screen. With social interactions shifting more into digital spaces, we wanted to give people an assistant that helps them express themselves effectively, understand others' emotions, and build deeper connections. We envisioned MoodScribe as a real-time conversational sidekick, there to help users navigate the nuances of digital interactions with confidence and empathy.

What it does

MoodScribe is an intelligent, emotion-driven social assistant that integrates seamlessly with platforms like Instagram DMs. It offers features like:

Real-Time Topic Suggestions: Suggests relevant topics based on the flow of conversation, helping users keep interactions smooth and engaging.
Tone Matching: Adjusts suggestions to match the conversation’s tone, whether it’s upbeat, serious, or empathetic.
Memory Recall: Builds rapport by recalling past conversation details and context, making interactions feel more personal and attentive.
Progress and Goal Tracking: Gives users insight into their social interactions, highlighting areas to improve, like asking more open-ended questions to foster meaningful dialogue.

Through these features, MoodScribe not only makes online conversations smoother but also helps users develop their social skills.

How we built it

MoodScribe was developed with a combination of front-end, back-end, and AI-powered APIs:

Frontend: Designed a sleek and responsive interface in HTML, CSS, and JavaScript, giving it a futuristic, cyberpunk aesthetic that reflects the advanced AI at work. The UI is crafted to be intuitive, with quick-access buttons for generating and editing responses.
Backend: Using Python and Flask, we connected the front-end with APIs and databases, handling login, session management, and data requests. Flask facilitated real-time interactions between the user and MoodScribe’s AI engine.
Meta API Integration: Integrated with Meta's APIs to enable seamless access to Instagram DMs, giving users a unified experience for managing conversations.
NLP and Emotion Analysis: Incorporated NLP models to analyze message tone and content in real-time, providing users with relevant suggestions and insights.

The project required strong coordination between front-end design, back-end integration, and AI implementation, each part working in harmony to deliver a cohesive and helpful user experience.

Challenges we ran into

Throughout the project, we encountered several challenges:

API Limitations: Meta’s API posed limitations on session management and data access. We had to carefully manage session IDs and permissions to ensure reliable, secure data handling.
Fine-tuning Implementation: We struggled when looking for resources to train and run inference on our model. Due to limited RAM we decided to utilize Unsloth as a way to optimize our training and inference procedures.
Creating an Intuitive UI: Designing a UI that was both aesthetically pleasing and user-friendly posed an interesting challenge, as we wanted MoodScribe to feel engaging yet straightforward to use. We decided on an Inside Out theme as it matched the goal of determining the mood of the conversation through sentiment analysis.

Accomplishments that we're proud of

We’re proud of building a tool that brings out the best in digital communication. From achieving accurate emotion detection to creating a memorable UI experience, we believe MoodScribe can positively impact how people interact online. Being able to manage real-time data from conversations and provide relevant, empathetic suggestions is an achievement we’re thrilled about. Additionally, we were able to successfully train and finetune our first model and the resulting loss was very close to 0.

What we learned

This project was a deep dive into real-time user interaction, API integration, and the nuances of NLP in social contexts. We gained valuable insights into how subtle differences in wording and tone can change a conversation's direction. It also taught us the importance of privacy and security when handling user data, especially with a conversational tool.

What's next for MoodScribe

Our vision for MoodScribe is constantly evolving! Next steps include:

Expanded Platform Support: Integrating MoodScribe with more platforms to expand its reach and impact.
Enhanced AI Capabilities: Improving MoodScribe’s emotional intelligence and tone matching for even more natural conversations.
User Customization: Allowing users to customize responses, tones, and suggested topics to make MoodScribe a tool for them to improve their social interactions.