Can you calm me?

The main scene with angry wife scenario.
Angry mom

Inspiration

As a non-native English speaker from Indonesia who hadn't used the language for years, I found it incredibly difficult to regain my speaking skills. This was frustrating because I saw so many tech opportunities I could explore outside of my country. To improve, I tried every approach: I hired tutors, used apps like ChatGPT and Character.AI, Duolingo—you name it. Then, while practicing with my daughter, I realized one crucial thing was missing: characters. The engaging, lifelike 3D characters that (almost) everyone loves. Over the last two years, I've built numerous production grade chatbot apps (audio and video) integrating LLMs. Now, I'm channeling that experience into creating a truly unique version that I'm excited to share with everyone.

What it does

User can do voice chat with 3d characters through scenarios or "scenes". There are two types of conversations. Challenge and Chill Lounge. In challenges they will face an angry 3d character based on the scenario / scene and trying hard to calm them down in 5 minutes, while in chill lounge the character will try to calm the user.

How I built it

I'm using react, vite, three.js and r3f for the 3d scene (environment, the character model and their expressions). Using Elevenlabs conversational 2.0 for best conversational experience.

Challenges we ran into

Bolt.new is great at building all the UI, including the 3D scenes (unexpectedly), but prompting for the camera, position, and character is surely super challenging (or maybe it's just my skill issue). And the hardest part is the lip sync of the character.

Accomplishments that we're proud of

The character looks alive enough and that makes me happy. From the breathing part, the movement and expression and how it interacted with the conversation system, the experience of talking with them, it just amazing. Please do try it.