Inspiration
Healthcare in Bharat is not limited by the lack of doctors — it is limited by access, complexity, and digital literacy.
Most apps assume patients can type symptoms, navigate menus, and understand medical language.
But for millions of families, healthcare begins with a simple voice question:
"Doctor, what should I do?"
We were inspired to build REX Healthify AI as an ambient, voice-first healthcare operating system that makes care accessible through natural conversation in Indian languages.
What it does
REX Healthify AI is a Gemini-powered autonomous health assistant that enables:
- Multilingual voice-first triage (Hinglish + dialect-friendly)
- Emergency detection + Golden Hour workflows
- Multimodal health intelligence, including:
- Plate Scan for nutrition understanding
- Vision-based vitals estimation (rPPG-inspired)
- Plate Scan for nutrition understanding
- Tool-based appointment booking through UHI-inspired workflows
- ABHA-linked identity and consent-based health continuity
- Automatic SOAP note generation from every interaction
- A Hospital Dashboard for clinics to manage patients, reports, and follow-ups
- Google Calendar tool calling to schedule appointments and reminders automatically
REX goes beyond chat — it executes real healthcare actions.
How we built it
We built REX using the Google Gemini ecosystem with an agentic architecture:
- Gemini 3 API for reasoning, triage, and structured clinical outputs
- Gemini Multimodal Vision for plate scanning and health understanding
- Agent orchestration + tool calling using Google ADK-style workflows
- React-based Hospital Web Dashboard for provider-side coordination
- Firebase + Cloud backend for authentication and real-time sync
- Knowledge grounding through Neo4j-based Medical RAG
The system operates as a multi-agent health OS: Triage Agent, Booking Agent, Vitals Agent, Literacy Agent, and Emergency Agent.
Challenges we ran into
- Designing low-latency, voice-first healthcare UX
- Keeping multimodal medical reasoning safe and grounded
- Handling asynchronous booking flows naturally in conversation
- Building both patient-facing and hospital-facing layers together
- Structuring tool execution beyond chatbot responses
Accomplishments that we're proud of
- Built a complete Discovery → Care → Booking → Hospital Management loop
- Implemented real agentic tool calling, not just conversational AI
- Created a Bharat-first healthcare OS with voice + visual literacy
- Integrated Google Calendar automation for real-world care execution
What we learned
We learned that the future of healthcare AI is not just intelligence — it is execution.
Multimodal reasoning, tool calling, and agent orchestration can transform healthcare from reactive systems into proactive, continuous care.
Most importantly, accessibility in Bharat requires voice, simplicity, and trust-first design.
What's next for REX Healthify AI
Next, we plan to expand REX with:
- Full ABDM/UHI sandbox integration
- Wearable-based continuous monitoring
- Multi-hospital deployments with analytics
- Preventive digital twin personalization
- Real-time emergency dispatch + family care networks
REX Healthify AI is a step toward an autonomous healthcare operating system for 1.4 billion people.
Log in or sign up for Devpost to join the conversation.