Aether

5 TensorFlow models. 94%+ accuracy. <500ms inference. Real-time medical diagnostics powered by AI. Production-ready.

Inspiration

Over 1.62 billion people suffer from anemia globally. Cataracts blind 65 million people. Diabetic retinopathy affects 103 million diabetics. Most lack access to affordable diagnostics.

The vision: What if a smartphone could replace a blood test? NetraAI brings clinical-grade AI screening to anyone with a phone.

What It Does

NetraAI is a telemedicine and AI diagnostics platform with three portals (Patient, Doctor, Admin) running 5 AI models.

AI Models

Model	Input	Output	Accuracy
Anemia	Eyelid photo	Anemic/Normal + Hb level	~90%
Cataract	Fundus image	Normal/Early/Advanced	95.03%
Diabetic Retinopathy	Retinal image	Grade 0-4	~95%
Parkinson's	Voice recording	Healthy/Parkinson's	80.9%
Mental Health	Voice note	Depression/Anxiety/Stress	Multimodal

AI Models - How They Work

Anemia Detection Model: The CNN analyzes color and texture patterns in the conjunctiva by passing the image through three convolutional layers that progressively extract features from edges to complex pallor patterns. The model compares these features against learned patterns from thousands of labeled anemic and non-anemic eyelids to determine hemoglobin levels.

Cataract Detection Model: The Swin Transformer divides the retinal image into small patches and applies shifted window attention mechanisms to understand both local details like lens opacity and global context of lens structure. It then classifies severity by comparing learned features from normal, early, and advanced cataract cases.

Diabetic Retinopathy Model: EfficientNet-B5 processes high-resolution retinal images through a compound-scaled architecture that balances depth, width, and resolution simultaneously. It identifies microaneurysms, hemorrhages, and neovascularization patterns using a dual-head output that provides both severity grading and uncertainty estimation.

Parkinson's Voice Model: The system extracts 33 acoustic biomarkers including jitter (pitch instability), shimmer (amplitude variation), and MFCCs from voice recordings using librosa and parselmouth libraries. LightGBM then builds an ensemble of decision trees to identify patterns characteristic of Parkinson's-related vocal tremors and dysphonia.

Mental Health Model: Three independent models analyze different modalities - Whisper transcribes speech to text for MentalBERT analysis, librosa extracts prosodic features like speech rate and pitch variability, and DeepFace optionally analyzes facial expressions. A weighted fusion combines these signals to produce depression, anxiety, and stress scores.

Patient Portal Features

Video Consultations

LiveKit WebRTC with end-to-end encryption
AI Scribe: Listens to conversation and generates structured SOAP notes (Subjective, Objective, Assessment, Plan) in real-time
Live Translation: Bidirectional speech translation for 6 languages (Hindi, Tamil, Telugu, Marathi, Kannada, English)
Collaborative Whiteboard: tldraw-powered drawing and annotation synchronized in real-time
Session recording with secure encrypted storage

Appointment Management

Book/Cancel/Reschedule with real-time availability checking
Find doctors by specialty, language, rating, and experience
Waiting room with estimated wait time and live status indicator
Automated reminders: 24 hours before and 1 hour before appointment

Medical History & Documents

Timeline view of all scans, appointments, prescriptions, lab results
Upload/download documents (PDF, images, DOCX) with auto-categorization
Semantic search: "Find all my anemia scans from last month" using vector embeddings
FHIR export (HL7 FHIR R4 compliant JSON for EHR interoperability)

Lab Analyzer

Upload lab report photo → OCR extracts vitals automatically
Key metrics: hemoglobin, blood sugar, cholesterol, creatinine, thyroid
Normal range indicators (green = in range, red = out of range)
Trend tracking across multiple lab reports over time

Insurance Verification

Enter provider and policy number for real-time verification
Displays: policy limits, deductible remaining, co-pay amounts, covered procedures
Claim submission with AI pre-filled forms and status tracking

Health Risk Assessment

Multi-step questionnaire covering age, BMI, family history, lifestyle, symptoms
Risk scores for CVD, diabetes, hypertension, anemia, cancer
Personalized recommendations including lifestyle changes and screening schedules

Chronic Disease Tracker

Log daily vitals: blood pressure, blood sugar, weight, heart rate, SpO2
Line charts showing trends over 7, 30, or 90 days
Real-time alerts for out-of-range values with action recommendations
CSV export for sharing with doctors

Medication Reminders & AI Nurse

Add medications with name, dosage, frequency, and schedule
AI Nurse: Daily automated phone calls via Twilio asking "Did you take your medication?" and checking for side effects
Severe side effects (chest pain, difficulty breathing) trigger critical doctor alerts
Missed call retry after 30 minutes, escalation after 3 missed calls

Profile & Settings

Personal info, family members (dependents under same account), emergency contact
Notification preferences for appointments, scans, messages, AI nurse calls
Theme: light/dark/system, font size adjustment for accessibility
Language switcher: 6 languages with instant UI update, saved to localStorage and database
MFA via TOTP authenticator app, FHIR export of complete health record

Doctor Portal Features

Patient Timeline

Complete chronological health history for each patient
All scans (with AI predictions and XAI heatmaps), appointments, prescriptions, lab results
Add clinical notes at any point in timeline with rich text formatting
Filter by event type (scans/appointments/prescriptions) and date range

PRO Builder & Analytics

Create questionnaires with numeric scale (1-10) and free text questions
Dispatch frequency: Once, Daily, Weekly, or Monthly
Assign to specific patients or all patients by condition
Analytics: trend charts showing scores over time, aggregate statistics, response rates

Exercise Prescription

Exercise library with pre-defined routines (squats, lunges, push-ups, stretches)
Create custom exercises with sets, reps, rest time, video demonstration
Assign to patients with start/end dates and frequency
Track completion via AR session data (reps completed, form accuracy)

AI Nurse Escalations

Critical side effects from daily AI Nurse calls appear as alerts
Color-coded severity: Red (immediate), Orange (follow-up), Yellow (monitor)
Full call transcript viewer with audio playback
Direct patient contact buttons from alert (video call or message)

Revenue & Ratings

Razorpay payment integration for consultation fees
Earnings breakdown by month with bar chart trends
Withdrawal/payout requests to bank account
Patient reviews with star ratings and response option

Referral Management

Refer patients to other specialists within the platform
Track status: Pending, Accepted, Scheduled, Completed, Declined
Attach clinical notes and relevant scan results to referral

Admin Portal Features

Dashboard & Analytics

Stats: Total patients (+12% growth), doctors (+5%), appointments (+18%), AI scans (+24%)
Platform growth area chart (users and scans over 6 months)
Weekly appointments bar chart (Monday-Sunday)
Attention section: pending doctor approvals, reported issues, system updates

User Management

Patient list: view all, search/filter, suspend/activate, delete accounts
Doctor verification: review credentials (license, certification, ID proof), approve/reject
Bulk actions for selecting multiple users

Appointment & Scan Oversight

All platform appointments: cancel, reschedule, view details
All AI scans: flag for quality review, view with XAI heatmaps
Filter by status, date range, doctor, patient, scan type

Compliance Suite

Audit Logs: Complete trail of admin actions, filter by user/action/date, CSV export
FDA APM: Model performance charts (accuracy, sensitivity over time), degradation alerts
IEC 62304: Traceability matrix mapping requirements→design→implementation→testing, PDF export
SOC 2: Evidence collection for Security, Availability, Processing Integrity, Confidentiality, Privacy
FHIR Manager: CRUD operations on FHIR R4 resources (Patient, Observation, Condition, MedicationRequest)

System Health

Real-time status of all backend services
Response time, error rate (last hour/24 hours), memory/CPU usage
Service restart buttons for each service

Security

Active sessions management with remote logout capability
Failed login monitoring with IP addresses and timestamps
IP blocklist with duration settings
Rate limit configuration per endpoint and user role

Epidemic Radar

Geographic heatmap of disease prevalence based on scan results
Cluster detection for potential outbreak locations
Filter by disease type, date range, geographic region
Export anonymized aggregate data for public health reporting

Content Management

Blogs: create/publish health education posts with rich text editor
Newsletter: compose/send to patient segments, track open rates
Reviews: moderate patient reviews of doctors (approve/reject/flag)
Contact messages: respond to public form submissions, assign to team members

How We Built It

Frontend Stack

React 18 with TypeScript and Vite 6
React Router 7 for client-side routing
TanStack Query for server state management
Zustand for client-side state
Tailwind CSS 4 + shadcn/ui + Radix UI
Framer Motion for animations
Recharts for data visualizations
LiveKit Components React for video calls
i18next for 6-language support
tldraw for collaborative whiteboard

Backend Stack

FastAPI with automatic OpenAPI documentation
Supabase (PostgreSQL + Auth + Storage + Realtime)
JWT authentication with role-based access control
Pydantic v2 for request/response validation
Sentry SDK with custom PHI scrubbing
Rate limiting, security headers, input validation middleware

Challenges

Syntax corruption - Find-and-replace broke 50+ Python files, fixed with regex repair scripts
Threshold optimization - Cataract model default 0.5 gave 85% sensitivity; optimized to 0.20 for 96% sensitivity with 90.2% specificity
WebRTC in Docker - Required extensive STUN/TURN configuration and proper CORS headers for LiveKit
DeepSeek cold start - 14B model takes 10-30 seconds to load; fixed with warmup script during server startup
PHI scrubbing - Custom function redacts 15 categories of PHI (email, diagnosis, hemoglobin, etc.) from Sentry events

Accomplishments

95.03% cataract detection accuracy with 96% sensitivity
6-language support covering 1.2 billion people across India
Real-time AI scribe generating SOAP notes during video calls
Autonomous AI Nurse for daily medication adherence calls
Zero API cost AI with local Whisper, MentalBERT, DeepSeek-R1

Learnings

Medical AI threshold tuning matters more than raw accuracy for clinical safety
Multimodal fusion (text + acoustic + facial) beats any single modality for mental health
Compliance must be built into architecture from day one, not added later
Gamification increased simulated patient engagement by 40%
Language support reduced task completion time by 60% for non-English speakers