Talk To Totoro

Talk To Totoro uses facial emotion recognition to monitor your mood and provide instant, personalized counseling advice to help you navigate your day.

Comment

Inspiration

Our team grew up with the magic of Studio Ghibli, and we wanted to bridge the gap between that nostalgia and modern technology. Talk To Totoro was born from a simple question: What if the forest guardian wasn't just a character on a screen, but a companion you could actually talk to? We wanted to create an AI experience that didn't feel like a sterile chatbot, but rather a warm, whimsical presence that mirrors Totoro’s gentle and supportive nature.

What it does

Talk To Totoro is an interactive AI counselor companion. Using high-latency multimodal capabilities, users can talk to Totoro to share their day, ask for advice, or simply enjoy a moment of calm. Totoro responds with a distinct personality—wise yet playful—and can even "sense" the user’s mood through natural language processing, offering comfort or celebration through Ghibli-inspired wisdom.

How we built it

We integrated the Gemini API as our core logic engine. Its API allows users to talk to the chatbot as it if were a therapist, while Deepface, analyzes facial expressions to give a more personalized experience. All coding was done in VS Code, utilizing GitHub Copilot to accelerate our frontend development. ChatGPT and Gemini helped us brainstorm the initial project structure and troubleshoot complex logic errors.

Challenges we ran into

One of our biggest hurdles was integrating our backend with our frontend code. This required several iterations of "vibe-coding" and prompt engineering as there were numerous bugs and errors.

Accomplishments that we're proud of

  • Authentic Persona: We are incredibly proud of how "Totoro" sounds. Through careful prompt engineering with Gemini, we moved past generic AI responses to create a character that feels genuinely supportive.
  • Seamless Integration: Getting our GenAI stack—Gemini, Copilot, and ChatGPT—to work in harmony allowed us to build a functional application in just 24 hours.

What we learned

We learned that the "personality" of an AI is just as important as its intelligence. This weekend taught us how to leverage Gemini's multimodal capabilities to handle emotional context rather than just factual data. We also gained a much deeper understanding of the GitHub ecosystem for managing rapid-fire team contributions.

What's next for Talk To Totoro

We want to take the "multimodal" aspect even further. Our next steps include:

  • Voice Integration: Using the speakers so Totoro can "hear" and "talk" to you!
  • IoT Connectivity: Connecting Totoro to smart home devices—imagine Totoro "growing a forest" by turning on your smart lights and plant humidifiers.
  • Expanded Lore: Adding more characters from the Ghibli universe like the Soot Sprites for a truly immersive experience
Share this project:

Updates