Trusai

Inspiration

In Pakistan, mental health is still a taboo topic. Many people feel uncomfortable or ashamed to visit a psychologist, and even when they do, it's hard to open up completely. Sometimes we hold things back because we’re afraid of being judged, or we just don’t know how to express what we’re feeling. That’s what inspired me to build Trusai — a space where people can talk freely without fear, and get some emotional support from a system that actually listens.

What it does

Trusai is an AI-based chatbot that acts like a friend. It listens to your voice and watches your facial expressions to understand how you’re feeling. Then it responds in a natural, caring way using both voice and text. It’s not just a normal chatbot — it understands emotions from your speech (using prosody features) and facial cues. It’s designed to help you let things out when you feel overwhelmed, without any pressure.

How we built it

The frontend was built using Next.js. When a user clicks “Start Call”, we access the webcam and mic. The audio is sent to Hume AI’s Speech Prosody model, which detects 48 emotional traits. The video frames are sent to a Flask backend running a PyTorch model trained to detect 7 facial emotions.

The Hume EVI (Empathic Voice Interface) is used to analyze the detected emotions and generate a response — both as text and as audio using TTS (text-to-speech). Everything runs in real-time to keep the conversation flowing naturally.

Challenges we ran into

The biggest challenge was integrating all the parts together in real-time — especially sending live audio and video data to two different models and syncing their outputs. Also, using the Hume API for both prosody and EVI took some debugging and careful timing. Facial emotion detection also needed to be optimized so that it wouldn't lag during live video.

Accomplishments that we're proud of

We were able to create a working system that can actually listen, understand emotion, and reply naturally. Seeing the first full emotional interaction — where the user said something sad and Trusai replied with care — was a proud moment. Also, the real-time syncing of all parts (audio, video, API response, frontend display) felt like a big step.

What we learned

I learned a lot about handling live data streams (audio/video), working with external APIs like Hume, and making a frontend that connects smoothly with a backend Flask API. Emotion detection from voice and face was new to me, and I also got to understand how important timing and response flow is for natural conversation.

What's next for Trusai

I want to add a custom-trained emotional memory module so that Trusai remembers past conversations. I also plan to improve the UI, add an avatar for the AI, and maybe support more languages. In future versions, I’d also like to integrate offline support so people in areas with limited internet can still use it.

Built With

• flask
• humeai
• javascript
• nextjs
• python
• pytorch