Hessonite

💡 Inspiration

In high-volume emergency rooms, doctors are forced to make life-or-death cardiovascular decisions using heavily fragmented data. A patient's health is often scattered across physical ECG printouts, siloed vitals, and incomplete historical records.

Furthermore, we noticed a glaring issue with existing cardiovascular AI models: Algorithmic Bias. Traditional models often perform significantly worse on underrepresented demographics (such as women), and they completely crash if certain data points (like a patient's historical EHR) are missing. We wanted to build a system that was not just accurate, but resilient, equitable, and transparent.

⚙️ What it does

Omni-Fusion is a multimodal, patient-first AI diagnostic platform that predicts cardiovascular risk. It unifies three distinct data streams:

1. Real-time 12-Lead ECG Waveforms
2. Immediate Clinical Vitals
3. Longitudinal Patient History

Instead of giving patients and doctors a confusing "black-box" risk number, Omni-Fusion provides Explainable AI (XAI). It generates clean, readable SHAP waterfall charts that explain exactly which vital signs and historical factors are driving the patient's risk score, empowering them to take control of their health.

🛠️ How we built it

We engineered a highly scalable, secure, and modern tech stack:

• Frontend: Built with Next.js 16 (App Router) and TailwindCSS, utilizing Recharts and D3.js to render aesthetic, high-contrast dashboards and patient insights.
• Backend Inference: Served via a lightning-fast FastAPI (Python 3.13) REST API, adhering to strict Pydantic v2 schemas.
• AI Architecture: The core OmniFusionNet is built in PyTorch. It utilizes a 1D ResNet to process raw ECG waveforms, a Deep MLP for vitals, and a GRU to process sequential patient history. These are combined using a Late-Fusion architecture.
• Database & Security: Powered by Supabase (PostgreSQL). We implemented strict Role-Based Access Control (RLS) to securely link patients to their doctors, mimicking FHIR schemas and ABDM-ready interoperability standards.

⚠️ Challenges we ran into

Our most significant challenge was Missing Modalities. In low-resource clinics, patients often arrive with no historical records. Traditional AI models fail when faced with null inputs. To fix this, we implemented Dynamic Routing via Masking during training. The network dynamically learns to bypass the GRU history branch and heavily weight the ECG and Vitals branches if history is missing, ensuring 100% uptime.

Our second major hurdle was Demographic Divergence. We noticed our baseline model had a 12.5% error divergence between genders. We solved this by engineering a custom Equity-Aware Loss Function that penalizes the network if error rates diverge between subgroups:

$$ \mathcal{L}{total} = \mathcal{L}{BCE}(y, \hat{y}) + \lambda \sum_{g \in G} \left| \mathbb{E}[\text{error} \mid g] - \mathbb{E}[\text{error}] \right| $$

By integrating this custom regularization term, we successfully reduced the demographic error gap from 12.5% to under 1.5%.

🏆 Accomplishments that we're proud of

• Achieving incredibly high inference accuracy across multimodal data inputs.
• Successfully integrating SHAP (KernelExplainer) to provide complete, human-readable transparency for our complex neural networks.
• Designing a frictionless, beautiful user interface that turns intimidating medical data into a comforting, empowering experience for patients.
• Ensuring the architecture is FHIR-compliant and primed for real-world national health integrations.

📚 What we learned

Building Omni-Fusion was a massive learning curve. We learned how to synchronize different temporal data scales (static vitals vs. continuous waveforms) into a single PyTorch computational graph. We also deepened our understanding of Explainable AI (XAI) and how crucial algorithmic equity is in modern HealthTech. On the frontend, we mastered handling complex data visualizations in Next.js.

🚀 What's next for Omni-Fusion

The next step is real-world clinical validation. We plan to expand the data pipeline to ingest live streaming data directly from wearable devices (like smartwatches). Ultimately, we want to deploy Omni-Fusion into low-resource triage centers where an instant, equitable, and resilient second-opinion can save thousands of lives.

Built With

• ai
• dl
• docker
• ml
• nextjs
• python
• react
• scikit-learn
• tailwind
• typescript