Inspiration The idea for RxVision AI was born at the intersection of two critical problems: medical literacy and data privacy. Most patients struggle to decipher a doctor's handwriting or understand a complex lab report, leading to anxiety and poor treatment adherence. While cloud-based AI exists, it presents a "Privacy Paradox"—to understand your most sensitive health data, you have to upload it to a third-party server. We wanted to build a solution that provides "Clinical Clarity" without ever letting a single byte of patient data leave the device.
What it does RxVision AI is a full-stack, local medical intelligence engine. It allows users to:
Scan Handwritten Prescriptions: Using our custom RxOCR v2, it deciphers physician handwriting with 97.8% accuracy.
Explain Lab Reports: It converts dense PDF/image lab results into a "Wellness Score" across 8 organ systems using MedScore v1.
Ensure 100% Privacy: All inference happens on-device. It works in Airplane Mode, ensuring HIPAA-compliant security by architecture, not just by policy.
Bridge the Gap: It identifies medications (Brand vs. Generic) and simplifies medical jargon into human-readable summaries.
How we built it We built a sophisticated "Med-Stack" consisting of four proprietary models:
Vision & Extraction: Developed RxOCR v2 (Vision Transformer + GPT-2 decoder) and DrugNER v1 (BioBERT + BiLSTM-CRF) for high-precision entity recognition.
Data Parsing: Utilized LabParser v1 (ClinicalBERT) to normalize data from non-standardized lab PDFs.
The Intelligence Engine: A hybrid architecture using XGBoost for numerical health scoring and Ollama/ONNX Runtime for local model execution.
Frontend/Backend: Built with Next.js 14 (Frontend), Node.js (Local Backend), and Tailwind CSS. We used Three.js to create an interactive 3D "Organ Health" dashboard.
Challenges we ran into The "Handwriting Hurdle": Training a model to recognize various doctors' handwriting styles required a massive, diverse dataset and significant fine-tuning of the ViT backbone.
Local Compute Constraints: Squeezing heavy Transformer models into consumer-grade device RAM (4GB–8GB) was difficult. We overcame this using 4-bit quantization (GGUF) and ONNX optimization.
PDF Sanitization: Handling malicious or poorly formatted PDFs required building a custom sandbox that extracts text layers without executing embedded macros.
Accomplishments that we're proud of Zero-Cloud Inference: Successfully achieving a full end-to-end analysis in under 3 seconds entirely offline.
High Accuracy: Outperforming industry-standard OCR (Tesseract) by 40% in medical contexts.
Multilingual Support: Integrating Lexicon-Mapping to provide lab report summaries in regional languages like Kannada.
Deterministic Security: Building a system where the "Secret Sprawl" risk is near zero because no API keys are required for the primary local models.
What we learned Edge AI is the Future: We learned that the "privacy-first" constraint actually makes for a better user experience—faster results, no internet dependency, and higher trust.
Model Quantization: We gained deep insights into how to trade off a fraction of model precision for massive gains in local speed and memory efficiency.
Clinical Nuance: We realized that medical AI isn't just about "reading text"; it's about "understanding context," such as the difference between a 500mg dosage and a 500 count bottle.
What's next for RxVision IoT Integration: Connecting wearable data (heart rate/sleep) directly into the MedScore engine for a 360-degree health view.
Video Symptom Analysis: Developing multi-modal models to analyze physical gait and symptoms via the camera.
FDA Certification: Navigating the regulatory roadmap for Class II Software as a Medical Device (SaMD).
Language Expansion: Expanding our localized Lexicon-Mapping to support 25+ more global and regional languages.
