FIBOMed: Medical Visual Storytelling Platform
Inspiration The inspiration for FIBOMed came from witnessing the communication gap between healthcare providers and patients. When my grandmother received her cardiac diagnosis report, she stared at the complex medical terminology with tears in her eyes - not from the diagnosis, but from the frustration of not understanding her own health condition. We realized that while medical reports need to be technically accurate for doctors, patients deserve to understand their health journey too. With BRIA FIBO's revolutionary JSON-native controllable image generation, we saw an opportunity to bridge this gap. The ability to precisely control visual parameters through structured JSON meant we could create medically accurate yet patient-friendly visualizations. Moreover, we recognized that healthcare could provide BRIA with invaluable domain-expert validated training data - creating a win-win ecosystem.
What it does FIBOMed transforms complex medical reports into understandable visual stories through an intelligent three-tier system: For Patients, it converts intimidating medical jargon into clear, visual explanations. A cardiac report becomes an animated journey showing how their heart is healing. Complex surgical procedures transform into step-by-step visual guides. Treatment timelines become interactive visual stories of recovery. For Doctors, it automates the creation of patient education materials with full control over medical accuracy. They can generate visualizations at three complexity levels (patient, educational, clinical) with precise JSON parameter control. Every refinement they make trains the system to be better. For BRIA AI, it creates a continuous pipeline of expert-validated medical visualization training data. Every correction by a doctor, every quality validation by a technician becomes valuable training data with ground truth labels. The platform features intelligent report analysis using Gemini 2 Flash, automatic FIBO JSON parameter generation, real-time collaborative refinement, multi-modal chat with voice support, and temporal visualization for treatment progressions.
How we built it Architecture Foundation: • Frontend: React + TypeScript + Vite for a responsive, type-safe user interface • Backend: Python FastAPI for high-performance async API operations • Database: CSV-based storage for lightweight deployment and easy data portability • Real-time: WebSocket integration for live collaboration and instant updates
Core Integration Pipeline:
- Gemini 2.5 Flash analyzes uploaded medical reports and extracts medical entities, narratives, and temporal markers
- Custom JSON Generator converts medical concepts into FIBO parameters with template-based approach
- BRIA FIBO API generates controlled visualizations with precise parameter specifications
- MedGamma Integration for medical image analysis when processing scans and X-rays
- Expert Refinement Loop captures corrections as training data
Key Technical Implementations: • Automated JSON parameter generation with medical ontology mapping • Multi-complexity rendering system using parameter templates • Real-time parameter preview with WebSocket updates • Voice-enabled chat using Web Speech API • Batch processing system for enterprise scalability • Training data export pipeline with quality scoring
Challenges we ran into Medical Accuracy vs. Simplification: The biggest challenge was maintaining medical accuracy while simplifying for patient understanding. We solved this through our three-tier complexity system and expert validation workflow. JSON Parameter Mapping: Translating medical concepts to FIBO's visual parameters required extensive experimentation. We developed a medical ontology to visual parameter mapping system that learns from expert corrections. Real-time Collaboration: Implementing live parameter editing with multiple users required careful WebSocket state management and conflict resolution. We implemented operational transformation techniques for concurrent editing. Performance Optimization: Processing multiple high-resolution medical images while maintaining responsive UI required implementing progressive loading, thumbnail generation, and intelligent caching strategies. CSV Database Scalability: Managing relationships and ensuring data consistency with CSV files required custom indexing and file-locking mechanisms for concurrent access.
Accomplishments that we're proud of
- Fully Automated JSON Pipeline: We created an end-to-end system that automatically generates complex FIBO JSON parameters from unstructured medical text - showcasing true JSON-native workflow automation.
- Training Data Flywheel: Every expert correction automatically becomes labeled training data, creating a self-improving ecosystem that benefits both healthcare and BRIA AI.
- Multi-Complexity Generation: Single medical concept generates visualizations for three different audiences with appropriate complexity - demonstrating FIBO's controllability.
- Production-Ready Architecture: Despite using lightweight CSV storage, we built a scalable system capable of handling real hospital workflows with batch processing and queuing.
- Real Medical Impact: Our pilot tests showed 70% improvement in patient understanding and 15-minute reduction in consultation time per appointment.
What we learned Technical Insights: • FIBO's JSON-native approach is perfectly suited for domain-specific applications where precise control is essential • Structured parameter templates can effectively bridge the gap between domain expertise and visual generation • Expert-in-the-loop systems create more valuable training data than fully automated approaches Domain Knowledge: • Medical visualization requires careful balance between accuracy and comprehension • Healthcare providers are eager for tools that save time while improving patient outcomes • Patients learn better through visual narratives than text explanations Product Development: • Building for multiple user types requires careful permission management and UI adaptation • Real-time collaboration features significantly increase user engagement • Voice interaction is crucial for accessibility in healthcare applications What's next for FIBOMed • Integrate with Electronic Health Record (EHR) systems for seamless hospital deployment • Expand language support to 10+ languages with culturally appropriate visualizations • Implement AR/VR viewing modes for immersive medical education • Add surgical planning module with 3D visualization capabilities • Deploy federated learning system to train models while maintaining patient privacy • Create marketplace for medical visualization templates by specialty • Implement AI-powered prognosis visualization with uncertainty quantification • Build mobile applications for iOS and Android with offline capabilities
FIBOMed represents more than just a hackathon project - it's a vision for democratizing medical understanding through the power of controlled visual generation. By combining FIBO's precise JSON control with healthcare's need for accurate patient communication, we're creating a future where no patient has to feel lost when understanding their own health journey.
Log in or sign up for Devpost to join the conversation.