Inspiration
We wanted to solve one of the biggest gaps in healthcare today — accessibility, personalization, and timeliness of care. Existing apps often lack emotional intelligence, deep biometric integration, or seamless end-to-end treatment flows. Our goal was to create an empathetic, AI-driven health companion that understands users not just through words, but through voice tone, facial expressions, and health data — available to anyone, anywhere.
What it does
Listens to users through natural voice conversations.
Observes facial expressions to detect symptoms or emotional states.
Syncs with smartwatches to track blood pressure, heart rate, sleep, and more.
Offers AI-powered medical consultations using Gemini and Vertex AI.
Generates instant prescriptions and (in the future) enables 1-click medicine delivery through partner pharmacies.
Presents all health metrics in a unified dashboard for better monitoring and preventive care.
How we built it
Frontend: Flutter for a cross-platform mobile app (iOS & Android).
AI Backbone: Google’s Vertex AI and Gemini LLM for medical insights, conversational AI, and vision analysis.
Hardware Integration: Native APIs and plugins for camera, microphone, and smartwatch data (Apple Health, Google Health Connect).
Backend: Secure cloud infrastructure with REST APIs for health data storage and integration.
Future Integration: Pharmacy APIs (1mg, Practo, etc.) for seamless prescription fulfillment.
Challenges we ran into
Ensuring accuracy and safety in AI medical recommendations.
Integrating multimodal inputs (voice, video, biometric data) in real time without performance drops.
Meeting healthcare compliance standards like GDPR and HIPAA.
Designing an interface that is both powerful and user-friendly for people of all ages.
Accomplishments that we're proud of
Successfully integrated facial expression analysis with AI medical conversations.
Achieved full smartwatch health data sync, not just steps and activity.
Built a unified dashboard combining AI consultations, prescriptions, and real-time health monitoring.
Developed a roadmap for pharmacy integration, bridging the gap from diagnosis to doorstep delivery.
What we learned
The power of multimodal AI in creating a more empathetic and accurate healthcare assistant.
Best practices for secure, compliant health data handling.
How crucial UX design is in healthcare apps, especially for elderly or less tech-savvy users.
What's next for AI-Powered Medical Assistant App
Fine-tune AI models for specialized medical domains.
Integrate e-pharmacy partners for 1-click medicine delivery.
Expand language and regional support for global accessibility.
Add insurance and hospital record integration for a complete digital health ecosystem.
Introduce community health features like family dashboards and personalized health coaching.
Log in or sign up for Devpost to join the conversation.