PetTwin Care

AI-powered pet health platform leveraging Google Cloud AI, Vertex AI, and Gemini to deliver real-time diagnostics and personalized care recommendations.

Comment

Project Name: PetTwin Care
Tagline: Giving Your Pet a Voice Through the First AI Digital Twin.
💡 Inspiration
"Your pet can't tell you when they hurt. I built something that can."
Veterinarians face a suicide rate 3-5x higher than the general population. The primary driver? Compassion fatigue from being unable to catch problems early enough. Meanwhile, pet parents miss early warning signs, leading to late-stage diagnoses and painful outcomes.
PetTwin Care creates a "Digital Twin" of each pet by continuously streaming wearable sensor data (accelerometer, gyroscope, heart rate, body temp) through Confluent's Kafka pipeline into Google Cloud Vertex AI for real-time anomaly detection. When the AI detects something abnormal, Gemini LLM generates a natural-language "Veterinary Triage" summary that's synthesized into audio via ElevenLabs—so your pet literally speaks to you about their health.
🏗️ How I built it
I treated this not as a simple app, but as an industrial IoT pipeline applied to biology.
The Nervous System (Confluent Cloud): The entire platform is built on Confluent's managed Kafka infrastructure. Every sensor event—heart rate spikes, movement anomalies, temperature changes—flows as a real-time stream through dedicated Kafka topics. This event-driven architecture replaces traditional REST APIs with a fault-tolerant, replayable data backbone that scales infinitely. Confluent Cloud handles partitioning, replication, and stream processing, making PetTwin's health monitoring system resilient and production-ready.
The Brain (Google Vertex AI & Cloud Run): A containerized Python microservice subscribes to Confluent Kafka streams. It runs a custom Anomaly Detection model (Isolation Forest) trained on the specific pet's history, running on Google Cloud Run for infinite scale.
The Voice (Gemini & ElevenLabs): When an anomaly token is generated, I feed it into Gemini Pro with a complex system prompt to generate a "Veterinary Triage" summary. This text is then synthesized into audio via ElevenLabs for a high-empathy user experience.
The Body (Next.js & Firebase): A beautiful, dark-mode frontend provides the user interface, utilizing Firestore for real-time document syncing.
🚧 Challenges I ran into
The "N=1" Problem: Generic breed data wasn't enough. A healthy 12-year-old Pug moves differently than a 2-year-old Husky. I had to implement an adaptive baseline system that "learns" the individual pet over the first 7 days, rather than relying solely on static thresholds.
Signal-to-Noise: Accelerometer data is noisy. Distinguishing between "playing fetch" and "erratic pain behavior" required fine-tuning my window functions in the data stream processing.
Latency vs. Cost: Chaining multiple AI models (Vertex -> Gemini -> ElevenLabs) initially caused delays. I optimized this by running the inference asynchronously via Cloud Pub/Sub, keeping the UI snappy.
🏆 Accomplishments that I'm proud of
Giving AI a Heart: I successfully moved beyond "cold data." The "Pet Voice" feature—where the app speaks to you as your pet's health advocate—creates an emotional bond that standard dashboards lack.
Real-Time Event Architecture: Successfully integrating Confluent Kafka proved I could build a streaming platform where every health metric becomes an event that can be processed, analyzed, and acted upon in real-time—transforming pet health monitoring from batch-based checks to continuous, proactive care.
🍀 What I learned
Context is King: Anomaly detection is useless without context. Gemini proved incredible at taking a cold number ("Activity Score: 40") and turning it into a contextual insight ("Max is 60% less
## Observability & Monitoring: Datadog Implementation
Our PetTwin Care platform includes a comprehensive observability stack using Datadog for real-time monitoring, dashboard analytics, and incident management. This enables reliable monitoring of system health, performance metrics, and quick incident detection.
### Datadog Observability Features:
1. Complete Monitoring Dashboard
- Real-time metrics tracking (CPU, Memory, Disk usage)
- Service health visualization across all components
- Custom widgets showing system load, request rates, and response times
- Multi-dimensional analysis with heatmaps, distribution charts, and time-series data
2. Intelligent Alerts & Monitors
- Threshold-based alerts for critical metrics (CPU >90%, Memory usage)
- Anomaly detection for unusual behavior patterns
- Forecast alerts to predict and prevent issues before they occur
- Severity-based alerting (SEV-1 Critical, SEV-2 High, etc.)
3. Application Performance Monitoring (APM)
- Python microservice instrumentation with real-time traces
- Integration with GitHub source code for error tracking
- Deployment tracking with Git commit metadata
- End-to-end request tracing across Kafka event streams
4. Incident Management
- Real-time incident creation and tracking
- Incident severity classification
- Automated command assignment
- Complete incident audit trails
### How to Access the Datadog Dashboard:
Demo Access Account:
- Email: judge-pettwin@example.com
- Password: ReadOnly123
- Access Level: Read-only (View dashboards, monitors, APM data, incidents)
Login Steps:
1. Visit: Datadog Shared Dashboard
2. Or login to https://datadoghq.eu with the credentials above
3. Navigate to "Dashboards" → "PetTwin Care - Complete Observability Dashboard"
4. View all metrics, monitors, and APM traces in real-time
What You Can Explore:
- ✅ System metrics dashboard with 20+ visualization widgets
- ✅ 9 active monitors covering CPU, Memory, Disk, and Anomaly Detection
- ✅ APM traces showing request flows through the Python backend
- ✅ Incident management system with active incident examples
- ✅ Service health overview and performance analyticsactive than his Tuesday average").
Cloud Native Power: Leveraging Google Cloud's managed services (Cloud Run, Firestore, Vertex AI) allowed me to build an enterprise-architecture in a weekend.
🍇 What's next for PetTwin Care
Vet Portal Integration: Allowing veterinarians to login and view the "30-Day Digital Twin History" before the patient even walks in the door.
Video Gait Analysis: Upgrading my prototype computer vision model to analyze user-uploaded videos for early signs of arthritis using Gemini Pro Vision.
Predictive Life Expectancy: Using BigQuery to analyze population health data and suggest lifestyle changes that could statistically add years to a pet's life.
Built With: google-cloud vertex-ai gemini cloud-run firestore bigquery cloud-functions firebase pub/sub python confluent-kafka next.js fastapi docker firebase

Built With

Share this project:

Updates