New Gen Talking tom
user getting banned
warning by smart tom ai
landing page
Voice to Voice Conversation
Use of fish audio and janitor AI

playlist?list=PL68NYUAEHLCEIboSYMv_06zzvv3w31Tgj&si=cgW3MckIhBjd-DN5

Inspiration

Multiplayer gaming is all about connection and fun, but toxic chat can ruin the experience. We've all seen it—arguments escalate, new players feel unwelcome, and moderators can't scale to every game session. We wanted to create an AI companion that could keep gaming communities positive, engaging, and safe without the need for expensive human moderation. Inspired by the nostalgic Talking Tom character, we built an AI moderator that's not just functional—it's genuinely fun to interact with.

What it does

Talking Tom Chat is an AI-powered chat moderator that game developers can integrate into their multiplayer games. Tom acts as a friendly companion who:

Moderates conversations - Detects and gently defuses tension before conflicts escalate
Engages players - Jumps into food discussions, tells personalized stories featuring all players, and playfully echoes messages
Welcomes newcomers - Greets new players and helps them feel included
Facilitates decisions - Helps groups make choices when discussions get stuck
Speaks naturally - Uses Fish Audio TTS to deliver emotion-rich voice responses that match the conversation's mood

Tom knows when to speak and when to stay silent, making him feel like a natural part of the conversation rather than an intrusive bot.

How we built it

Tech Stack:

Backend: Python FastAPI with Socket.IO for real-time WebSocket communication
AI Brain: Anthropic's Claude AI (Sonnet 4.5) with a carefully crafted system prompt that defines Tom's personality and behavior
Voice: Fish Audio TTS API for natural, emotion-tagged speech synthesis
Frontend: Next.js with TypeScript, React, and Tailwind CSS for a responsive, modern UI
Deployment: Backend on Render.com, Frontend on Vercel

Architecture:

Real-time chat messages flow through Socket.IO
Claude AI analyzes context and decides whether Tom should respond
If responding, Claude generates speech with emotion tags (e.g., (happy), (excited), (calm))
Fish Audio converts the tagged text into natural-sounding voice
Audio streams back to all players in the room

Key Features:

Priority-based response system (13 different triggers)
Context-aware conversation analysis
Emotion-tagged speech for authentic voice delivery
Real-time multiplayer room management
Cross-platform responsive design

Challenges we ran into

Getting the AI to "shut up" - Our biggest challenge was teaching Tom when NOT to talk. Early versions were too chatty. We solved this with a sophisticated priority system and a [[NO_OUTPUT]] token that tells Tom to stay silent.
Fish Audio integration - Working with emotion-tagged TTS was new territory. We had to carefully map conversation contexts to appropriate emotion tags and ensure the tags were properly formatted according to Fish Audio's strict requirements.
Real-time audio streaming - Serving audio files efficiently while maintaining low latency across Socket.IO connections required careful backend optimization and proper CORS configuration.
Deployment complexity - Managing separate frontend and backend deployments with proper environment variables, WebSocket support, and CORS policies across Vercel and Render took careful configuration.
Prompt engineering - Crafting a system prompt that balanced personality, helpfulness, and silence was an iterative process. We went through many versions to get Tom's voice just right.

Accomplishments that we're proud of

Created a genuinely useful tool - Tom isn't just a gimmick; he actually improves group chat dynamics
Achieved natural voice interactions - The emotion-tagged speech makes Tom feel alive and responsive
Built a production-ready API - The backend is scalable and ready for real game integrations
Designed an intuitive UX - Players can jump into rooms and start chatting immediately
Implemented smart context awareness - Tom understands ongoing conversations and responds appropriately
Successfully deployed - Full stack application running smoothly in production

What we learned

AI needs guardrails - Sophisticated gating logic is essential for AI companions to feel natural rather than annoying
Voice changes everything - Adding emotion-rich TTS transformed Tom from a text bot into a character people actually enjoy
Real-time is hard - WebSocket management, connection handling, and audio streaming require careful architecture
Prompt engineering is an art - Small changes in system prompts can dramatically affect AI behavior
User experience matters - Even powerful AI needs a great interface to be useful
Deployment strategy - Separating frontend and backend into dedicated platforms (Vercel + Render) simplifies scaling and maintenance

What's next for Tommy Chat

Short-term:

Game integrations - Partner with indie game developers to test Tom in real gaming environments
Enhanced personality modes - Let developers customize Tom's personality for different game genres
Analytics dashboard - Give developers insights into chat health and Tom's engagement metrics
Multi-language support - Expand beyond English to reach global gaming communities

Long-term:

Voice commands - Let players interact with Tom using voice input
Custom training - Allow developers to fine-tune Tom for their specific community culture
Sentiment analysis - Provide real-time toxicity scores and proactive moderation
Plugin marketplace - Create an ecosystem where developers can add custom Tom behaviors
Mobile SDK - Native iOS/Android libraries for mobile game integration
Advanced storytelling - Dynamic quest generation and interactive narratives featuring players

Vision: Transform Tommy Chat into the go-to AI companion for every multiplayer game, making online gaming communities safer, more fun, and more engaging—one conversation at a time.

Try it live: https://tommy-chat.vercel.app/ GitHub: https://github.com/Rushabh-Runwal/calhacks/tree/main Backend API: https://backend-talking-tom.onrender.com