NovaTriage

NovaTriage is a privacy-first AI triage copilot using Amazon Nova to turn voice, text, and documents into urgency scoring and a clinician-ready handoff in seconds.

Comment

Inspiration

Triage is one of the most critical steps in healthcare, but it often happens under time pressure with fragmented and unstructured information. Caregivers, EMS teams, and hospital intake staff must quickly interpret symptoms, patient history, and sometimes unclear documents or device readings. At the same time, many digital health tools either oversimplify triage or require sending sensitive patient information to the cloud without strong privacy guarantees.

NovaTriage was inspired by the idea that AI could help structure this chaotic intake process while keeping privacy at the center. We wanted to explore how multimodal AI and agent-based reasoning could assist caregivers, clinicians but also normal peaple, by rapidly organizing symptoms, detecting potential red flags, and generating a clear handoff summary—without attempting to replace medical judgment.

This is also intended to be a "home-assistant" tool that helps non-clinical people to better explain and trace symptoms.

Our goal was to build a system that accelerates understanding, not diagnosis, while demonstrating how modern AI models like Amazon Nova can support real-world healthcare workflows in a responsible way.

I strongly believe in this project but I know there is a lot of work to do. In my opinion is already a solid base that can be shared with normal people but maybe, it is better to plan an internal trial with home-assistant that can return feedback. Nova throught Bedrock was a really good way to integrate many AI capabilities and I think this project can help a lot of people with a little piece of code, without enourmous and difficoult to manage architecture.

What it does

NovaTriage is a privacy-first, mobile-first AI triage copilot powered by Amazon Nova models.

The application allows caregivers, EMS staff, or hospital intake personnel to:

  • Describe patient symptoms using voice or text
  • Upload medical documents or images
  • Automatically structure unorganized symptom descriptions
  • Detect potential red flags
  • Estimate an urgency level
  • Generate possible clinical clusters with confidence scores
  • Produce a clinician-ready handoff summary

A key feature is client-side anonymization: personally identifiable information is detected and redacted on the user’s device before any data is sent to the backend.

NovaTriage is also:

  • Multilingual (English default, Italian supported)
  • Protocol-aware, supporting different triage systems
  • Multimodal, combining text, speech, and documents
  • Mobile-first, designed for real field use

Additionally, it can help to:

  • Retrieve informations about drugs and dosage
  • Analyze clinical results (eg. blood analysis) to help understand what does it means
  • Assist users not so smart with tech with the voice assistant based on NovaSonic models
  • Retrieve urgency numbers based on your region

Instead of replacing doctors, NovaTriage helps clinicians understand the patient situation in seconds.

How we built it

NovaTriage was built as a mobile-first Progressive Web App (PWA) with a modular backend architecture.

Frontend

The user interface was built with:

  • Next.js
  • TypeScript
  • TailwindCSS
  • PWA architecture

The UI guides users through a structured triage flow:

  1. Case setup
  2. Symptom intake
  3. Attachments (images or documents)
  4. AI analysis
  5. Urgency result and clinician handoff summary

A client-side privacy engine runs before any backend request. It detects and anonymizes sensitive information such as names, emails, phone numbers, and identifiers, replacing them with anonymized patient aliases.

Backend

The backend is built with:

  • Node.js
  • Fastify
  • AWS Bedrock
  • Amazon Nova models

The system orchestrates a multi-agent AI pipeline, where each agent performs a specific task:

  1. Normalize symptom descriptions
  2. Detect language
  3. Extract structured clinical data
  4. Analyze multimodal evidence
  5. Detect safety-critical red flags
  6. Estimate urgency
  7. Generate clinician handoff summary

AI models

NovaTriage uses multiple Amazon Nova models:

Nova Lite

Used for lightweight reasoning tasks such as:

  • language detection
  • symptom structuring
  • clarification questions
  • safety validation

Nova Pro

Used for more advanced reasoning tasks:

  • multimodal analysis
  • differential cluster generation
  • clinician handoff summary creation

Nova Sonic

Used for voice-based assistant.

Infrastructure

The system runs locally using Docker containers and can be deployed to AWS using Helm charts.

Core services include:

  • Amazon Bedrock for AI inference
  • S3 for redacted documents
  • DynamoDB for anonymized case storage

Challenges we ran into

One of the biggest challenges was balancing AI reasoning with clinical safety. We needed the system to provide meaningful insights without behaving like a diagnostic engine.

To address this, we implemented a hybrid triage model that combines deterministic safety rules with AI reasoning. Critical red flags are detected using rule-based logic before AI reasoning is applied.

Another challenge was privacy-first architecture. Implementing client-side anonymization while preserving clinically relevant information required careful design. We needed to ensure that identifiers were removed without accidentally stripping useful medical context.

Handling multimodal input also presented difficulties. Medical information often comes from multiple imperfect sources—text descriptions, images, and scanned documents. Designing an AI pipeline that could combine these signals reliably required structured intermediate outputs and strict validation.

Accomplishments that we're proud of

We are especially proud of several aspects of NovaTriage.

First, we implemented client-side anonymization, meaning sensitive patient information is removed before any data leaves the device. This significantly improves privacy compared to many typical AI applications.

Second, we built a fully agent-orchestrated AI pipeline instead of relying on a single model call. Each agent performs a specialized task, which improves reliability and transparency.

Third, we created a clinician-ready handoff summary that condenses a complex intake into a short, structured format that can be understood in under 20 seconds.

Finally, the entire system runs as a mobile-first PWA with real AI inference, using Amazon Nova models for reasoning, multimodal understanding, and voice intake.

What we learned

Building NovaTriage reinforced how powerful agent-based AI architectures can be when applied to structured workflows.

Instead of relying on one large model call, dividing the workflow into specialized agents made the system easier to reason about, test, and validate.

We also learned that privacy must be a design principle, not an afterthought. Implementing anonymization on the client side required additional effort, but it significantly improved trust and safety.

Finally, we saw how multimodal AI can better reflect real-world scenarios. In healthcare contexts, symptoms, images, and documents often need to be interpreted together rather than separately.

What's next for NovaTriage

Future improvements could expand NovaTriage in several directions.

We plan to explore:

  • Integration with clinical triage protocols used by hospitals
  • Support for additional languages and healthcare systems
  • More advanced multimodal analysis for medical images
  • Integration with wearable device data
  • A clinician dashboard for case review and triage analytics

Longer term, NovaTriage could evolve into a privacy-first intake assistant that helps healthcare providers reduce intake time, improve triage consistency, and better understand patient conditions before the first clinical interaction.

Built With

Share this project:

Updates