SereneAI

Inspiration

In today’s fast-paced world, balancing mental well-being with productivity is a challenge for many. We were inspired by the idea of creating a tool that merges emotional intelligence with proactive productivity management. Our goal was to help users manage stress, monitor their emotional health, and stay on top of tasks in a personalized and seamless way. Serene AI was born from the desire to provide users with a voice-powered, emotionally aware companion that not only listens but also takes meaningful actions to support their well-being.

What it does

Serene AI is a voice-to-voice web application that assists users in managing both their mental well-being and productivity. It listens to users' thoughts, emotions, and concerns, offering thoughtful responses that include personalized advice and proactive actions. Through integrations with Gmail and Google Calendar, Serene AI helps users manage their schedules and tasks while simultaneously tracking emotional trends over time. It transforms well-being recommendations into actionable steps to improve overall wellness and productivity.

How we built it

Serene AI was developed using a robust, multi-layered tech stack:

Frontend: We created the interface with React and used Chart.js for visualizing emotional trends and mood scores over time.
Backend: Flask was utilized to manage API interactions and host the custom AI model, which is powered by Groq, with our embedding models from OpenAI.
Voice-to-Voice Interaction: Speech-to-text and text-to-speech capabilities were enabled using the VAPI API, allowing seamless voice interaction.
AI & Sentiment Analysis: We integrated Hume AI for advanced emotional sentiment tracking and analysis, while ChromaDB stores conversation histories and emotional data.
Productivity Management: A multi-agent framework was implemented to integrate Google Calendar and Gmail, enabling Serene to manage events, send emails, and adjust schedules based on the user’s emotional state and preferences.

Challenges we ran into

One of the major challenges was creating agents that could interpret natural language queries to control various APIs, such as Google Calendar and Gmail. Fortunately, Groq's function-calling models simplified this process, making it easier to handle complex requests and improve the system's functionality.

Accomplishments that we're proud of

We’re proud of creating a fully functional, voice-powered wellness coach that combines mental well-being support with productivity tools. A major achievement was building a custom LLM that understands and responds to users' emotions in real-time. The multi-agent framework, which allows Serene to proactively manage calendars and emails based on emotional input, is another significant accomplishment. Additionally, the ability to visualize emotional trends and provide users with actionable well-being insights over time is a feature we’re particularly excited about.

What we learned

Throughout the development of Serene AI, we deepened our understanding of how AI and emotional intelligence can intersect. We learned the importance of balancing personalization with user autonomy—ensuring that Serene’s suggestions are not only helpful but also actionable. We also enhanced our skills in integrating multi-agent systems, refining our ability to combine AI-driven emotional insights with practical productivity tools.

What's next for Serene AI

Looking ahead, we plan to:

Expand integrations with additional productivity tools, such as task management apps.
Improve our sentiment analysis with more sophisticated real-time mood prediction models.
Incorporate guided meditations, breathing exercises, and other wellness activities based on emotional feedback.
Add support for multiple languages to make Serene AI accessible to a global audience.

With these enhancements, Serene AI aims to become an even more effective personal wellness coach for users everywhere.