AGastIa

About the Project

Inspiration

We built AGastIa because medical reports are often full of information but empty of clarity. A patient sees numbers, acronyms, and dense pages; we saw an opportunity to turn that noise into a clear health story that people can actually understand and act on. Our goal was simple: make medical intelligence visible, not hidden behind jargon.

What I Learned

This project taught us how powerful it is when machine learning is not just accurate, but useful. We learned how to combine OCR, domain-specific LLMs, retrieval, visualization, and causal reasoning into one coherent product, and how to design AI that serves both clinicians and patients. Most importantly, we learned that the best ML systems are not just smart — they are explainable, trustworthy, and built for real-world decision-making.

How I Built It

We built AGastIa as a full-stack AI platform with a React and FastAPI foundation, then layered in a medical intelligence pipeline that goes from report upload to insight delivery. The system extracts text with OCR, identifies medical entities with a domain-tuned LLM, maps findings to organs, scores severity, generates dual summaries, and visualizes everything through an interactive 3D body model and causal graph. For long reports and scan analysis, we added RAG, MedSAM segmentation, and vision-based inference so the system can handle both structured labs and complex imaging workflows.

Challenges Faced

The biggest challenge was turning messy, multi-format medical data into structured, reliable outputs without losing clinical meaning. We had to balance precision with usability, manage long-context extraction, connect findings across organs, and keep the experience fast enough to feel polished. Another major challenge was building a system that is both technically ambitious and demo-ready — something that feels like a moonshot, but still works cleanly end to end.

🚀 Track Fit & Why We Stand Out

| Best Machine Learning Track Hack | We built a multimodal ML system combining OCR, NLP, vision models, and causal reasoning into a single pipeline that generates clinically meaningful insights — not just predictions. | | Best Data Visualization Experience | We go beyond charts with interactive 3D body visualization + causal graphs, turning complex medical data into intuitive, explorable insights. | | Best Champion of Open Source (Featherless) | Our system is built on and integrates open-source ML models and tools, demonstrating how community-driven AI can power real-world healthcare solutions. | | [MLH] Best Use of Gemini API | We leverage Gemini for advanced reasoning, summarization, and contextual understanding, enabling high-quality clinical and patient-friendly outputs. | | [MLH] Best Use of Auth0 | We implemented secure, scalable authentication with Auth0, including role-based access (patient/doctor/admin) — critical for real-world healthcare systems. | | [MLH] Best Domain Name (GoDaddy) | AGastIa is a distinct, memorable brand identity, aligned with healthcare intelligence, making the product recognizable and market-ready. | | Best Moonshot Hack (Bitcamp) | We tackle a massive, real-world problem — medical data interpretation — with an ambitious, end-to-end AI system that feels like a production-grade product. | | Best Bitcamp Hack | Our project combines technical depth (AI + systems) with real-world impact, delivering a polished, fully functional product — not just a prototype. | | Best UI/UX Hack (Bitcamp) | We designed a clean, intuitive experience that simplifies complex medical data through guided flows, visual storytelling, and explainable outputs. |