Inspiration
Have you ever watched Midnight Gospel or Rick and Morty and wondered just how far you could push the boundaries of storytelling and technical artistry? That's exactly what inspired me! I wanted to create a truly immersive experience with 3D characters having real-time, realistic conversations. Beyond the technical challenge, I was also driven by a desire to help combat loneliness and foster accountability in a way that feels genuinely fun and not, well, creepy. My girlfriend and 12-year-old step-daughter even helped me prototype, aiming for a younger teenage audience.
What it does
This project is a functional prototype of a real-time conversational experience with three-dimensional characters. Imagine being able to chat with a dynamic 3D avatar that not only speaks naturally but also moves its mouth and body in a realistic way as you interact. It's designed to bring a new level of immersion to digital conversations, making them feel much more authentic and engaging.
How we built it
Building this was a crash course in cutting-edge tech! A year ago, conversational AI was a nightmare, but with modern advancements, it became feasible. We leveraged Eleven Labs for incredibly realistic speech synthesis, making our characters sound alive. Supabase served as our robust database, keeping everything organized, and Bolt was a lifesaver in connecting all the pieces and making the flow smooth. On the visual side, creating the 3D characters was a first for me. With the amazing help of a friend who's a wizard with Blender, we sculpted our characters and then tackled the fun challenge of rendering those 3D files using Three.js.
Challenges we ran into
This project threw a lot of curveballs my way! My biggest hurdle was truly stepping into the world of 3D character creation. I pride myself on being technical, and this was definitely pushing my boundaries. Beyond just modeling, getting those subtle, natural mouth movements to sync perfectly with the speech, and making the characters move in an organic way,
Built With
- bolt
- elevenlabs
- react
- supabase
- three.js
- typescript
Log in or sign up for Devpost to join the conversation.