• 

  Project Logo

• 

  Home Page

• 

  Video Call Page

• 

  Emoji Slide

• 

  Community List

• 

  MongoDB Vector Search

• 

  Emotional Analysis Trained Model

Inspiration

During COVID-19, we witnessed elderly people struggling with digital communication - feeling isolated despite being connected. Traditional chat platforms failed to understand their emotional needs. The key insight: elderly users often struggle to express emotions clearly in text but show genuine feelings through facial expressions. We realized that combining both could create truly empathetic AI companions.

What it does

SilverLink AI is a real-time multimodal emotion analysis system that:

Analyzes text sentiment using fine-tuned DistilBERT + Google Cloud NLP Recognizes facial emotions through custom MobileNetV2 models Fuses both inputs using elderly-optimized algorithms Provides real-time emotion feedback for chatbots, virtual companions, and healthcare monitoring Enables intelligent user matching based on emotional compatibility

Core Innovation: Our fusion algorithm accounts for elderly communication patterns (like saying "I'm fine" while looking sad) and weights facial expressions higher than text for more accurate emotion detection.

How we built it

AI Pipeline

Text Analysis: Fine-tuned DistilBERT on elderly-specific language patterns + Google Cloud NLP for multilingual support Facial Recognition: Transfer learning with MobileNetV2, trained on elderly facial expression datasets Fusion Algorithm: Confidence-weighted combination with elderly-specific adjustments Backend Architecture

FastAPI microservices deployed on Google Cloud Run WebRTC signaling server for real-time video chat MongoDB Vector Search for intelligent user matching RESTful APIs ready for frontend integration

Real-time Integration

WebSocket-based communication for live emotion tracking Optimized inference pipeline achieving <1.5s response time Docker containerization for scalable deployment

python# Core API Example POST /api/analyze-emotion { "text": "I'm feeling okay today", "image": "base64_encoded_face_image", "context": "daily_check" }

Response: { "text_sentiment": "neutral", "face_emotion": "sad", "final_emotion": "concerned", "confidence": 0.84, "recommendations": ["offer_support", "gentle_conversation"] }

Challenges we ran into

Model Optimization: Large transformer models exceeded Cloud Run memory limits

Solution: Model distillation and pruning, reduced size by 60% while maintaining accuracy

Elderly-Specific Data Scarcity: Limited training data for elderly expressions

Solution: Transfer learning, data augmentation, and synthetic elderly emotion datasets

Fusion Algorithm Complexity: Text and facial emotions often conflicted

Solution: Developed "hidden emotion" detection for cases where elderly say "fine" but look distressed

Real-time Performance: Needed <2s response for live chat integration

Solution: Parallel processing, efficient model loading, and optimized inference pipeline

Accomplishments that we're proud of

What we learned

Multimodal AI requires cultural sensitivity: Emotion expression varies significantly across cultures and age groups Production AI ≠ Research AI: Models need significant optimization for real-world deployment User-centric design is crucial: Technology must adapt to elderly users, not vice versa Edge cases matter: Elderly communication patterns differ from typical AI training datasets

What's next for SilverGame

Immediate (Next 3 months)

Complete React frontend with real-time emotion visualization Unity 3D virtual companions that respond to user emotions Mobile app for elderly users with simplified interfaces

Future Vision

Speech emotion recognition for complete multimodal analysis Personalized emotion learning over time Clinical-grade integration for healthcare monitoring Global deployment with multi-cultural emotion understanding

Built With

• docker
• fastapi
• gcp
• javascript
• mongodb
• python
• pytorch
• react
• tailwind
• transformer-based-emotion-analysis
• vectorsearch
• webrtc