Moodify

Inspiration

In a world full of digital interactions, our emotional needs are often overlooked. We wanted to create a system that listens not just to what you say, but how you say it — and responds with empathy, intelligence, and a human touch.

Inspired by the challenge posed by Neuphonic, we imagined a future where voice interfaces do more than just respond — they understand, care, and support emotional wellbeing in real time.

What it does

🎙 Records your voice
🧠 Detects your emotional tone using a fine-tuned Hugging Face SER model
💬 Converts speech to text
🧘‍♂ Uses Gemini 2.0 to generate personalized, caring advice
🔊 Delivers that advice back to the user using Neuphonic's hyper-realistic Voice AI
🌐 Runs on a clean and responsive Streamlit UI

How we built it

Component	Technology
Speech-to-Text	SpeechRecognition
Emotion Recognition	Hugging Face Wav2Vec2 (SER model)
AI Advice Generation	Gemini 2.0 Flash (Google GenAI)
Voice Output	Neuphonic API + TTSConfig
Frontend	Streamlit
Audio Processing	Butterworth Filter, Noise Reduction (SciPy)
(Optional) Database	Firebase Firestore for storing logs

Challenges we ran into

🎧 Audio processing quirks: Handling microphone input and ensuring clean audio in noisy environments was tough. We solved this with custom filters and normalization.
💬 Integrating multiple APIs: Connecting Hugging Face, Gemini, Neuphonic, and Firebase together inside one real-time pipeline took careful coordination.
🌍 Multi-language support: Mapping audio models with the correct language recognition required trial and error.

Accomplishments that we're proud of

Built a functional Voice AI system from scratch in under 36 hours
Seamlessly integrated emotion detection, advice generation, and voice response
Created a calming, responsive, and actually useful experience
Designed a platform that's both technically robust and emotionally impactful 💛