ICU HITL Reinforced Agent: Accountable AI for Critical Care

“AI watches. Humans decide. The system learns.” — A human-in-the-loop clinical AI for safer ICU decisions.

Comment

Inspiration

ICUs generate enormous amounts of data — vitals, labs, ventilator settings, fluid balance — but clinicians must interpret all of it in real time, often under sleep deprivation and emotional strain. Despite more data and AI tools than ever, outcomes haven’t improved because existing systems are black boxes that clinicians can’t trust. We wanted to design an AI that thinks with doctors, not for them — one that reasons transparently, validates safely, and learns responsibly from human feedback.

What it does

The ICU HITL Reinforced Agent is a multi-agent, human-in-the-loop (HITL) decision support system built entirely on AWS Bedrock. It continuously analyzes patient signals through four lightweight agents — Vitals, Labs, Ventilator, and Fluid — then merges insights for reasoning with Amazon Nova Pro. Before any recommendation is shown, safety guardrails and policy validation run automatically. If risk is detected, the workflow pauses for human approval in Slack. Every decision is logged, auditable, and contributes to a self-improving loop that learns from rejections under full human supervision.

How we built it

Multi-Agent Orchestration: AWS Step Functions + nine Lambda agents (data → reasoning → validation → HITL → audit). LLM Reasoning: Amazon Bedrock Nova Pro with RAG over Sepsis-3, ARDSnet, and institutional guidelines. Safety & Policy: Bedrock Guardrails + Titan Text Express validation layer. Human-in-the-Loop: Slack integration via API Gateway for real-time approval or rejection. Audit & Learning: DynamoDB + S3 Object Lock (WORM) + CloudTrail ensure immutable traceability. Self-Improving Loop: Rejected recommendations are clustered daily, analyzed by Nova Pro, reviewed by humans, and converted into new operational rules.

Challenges we ran into

Designing a workflow that balances autonomy with accountability — ensuring no unsafe action bypasses HITL approval. Implementing Guardrails + JSON Schema validation across multiple agents without breaking latency targets (< 90 s p95). Building a self-learning system that can evolve safely without modifying model weights. Managing cost efficiency and concurrency within AWS service limits during orchestration testing.

Accomplishments that we're proud of

  1. From Black-Box to Accountable AI 　We transformed the traditional black-box AI into a transparent and accountable clinical system. Every recommendation now includes clear rationale, evidence citations, and full data lineage — enabling clinicians to see not just what the AI suggests, but why.
  2. Safety-First Architecture 　We built a multi-layered safety stack combining Bedrock Guardrails, Titan validation, and mandatory HITL approval for high-risk paths. 100% of critical recommendations are routed through human review before execution, ensuring trust and reliability.
  3. Local Adaptation 　Each institution’s unique protocols, devices, and formularies are encoded directly into the Knowledge Base and Guardrail policies. This allows the system to adapt seamlessly to local standards of care, rather than relying on one-size-fits-all models.
  4. Continuous Learning Loop 　Clinician rejections are automatically mined into structured insights, generating new operational rules reviewed by humans before activation. This creates a self-improving feedback loop that enhances precision over time — without costly retraining or model updates.

What we learned

How to orchestrate reasoning LLMs and rule-based agents inside a secure AWS VPC. The critical importance of human trust — explainability and evidence citations matter more than raw model accuracy. That safe autonomy requires feedback loops, not just outputs — AI must learn how it was wrong. Interdisciplinary teamwork between clinicians and engineers is essential for accountable AI in medicine.

What's next for ICU HITL Reinforced Agent: Accountable AI for Critical Care

EHR Integration (FHIR): connect with hospital systems for real patient workflows. Multi-site Federation: share operational insights while preserving local policies. Regulatory Alignment: expand HIPAA/APPI compliance documentation for real-world clinical pilots.

Built With

  • amazon-bedrock-(nova-pro
  • amazon-cloudfront
  • amazon-dynamodb
  • amazon-web-services
  • api-keys
  • aws-api-gateway
  • aws-cdk-v2
  • aws-cli
  • aws-cloudformation
  • aws-cloudwatch
  • aws-iam
  • aws-lambda
  • aws-step-functions
  • aws-x-ray
  • bedrock-guardrails
  • bedrock-knowledge-base-(rag)
  • cloudfront
  • cloudfront-cdn
  • cloudwatch-logs
  • cors
  • esbuild
  • event-driven-step-functions
  • framer-motion
  • git
  • https/tls
  • human-in-the-loop-workflow
  • iam-roles-&-policies
  • javascript
  • jest
  • json
  • json-schema
  • lucide-react
  • microservices
  • multi-agent-system
  • next.js-14
  • next.js-build
  • node.js
  • npm
  • pytest
  • python-3.12
  • python-pip
  • react-18
  • recharts
  • rest-api
  • route-53
  • s3-sync
  • serverless-architecture
  • slack-integration-(planned)
  • static-site-generation
  • tailwind-css
  • titan-embeddings)
  • titan-text-express
  • typescript
  • typescript-compiler
  • typescript-types
  • zod
Share this project:

Updates