Replay

Replay makes AI-assisted clinical decisions transparent, verifiable and replayable.

Comment

Replay

Inspiration

As AI becomes increasingly embedded in clinical workflows, one question becomes more important than ever:

How do we understand why an AI-assisted decision was made months after the event?

Replay was inspired by the idea that future healthcare systems need more than powerful AI—they need transparency, accountability and the ability to learn from every decision.

The project was also influenced by a personal experience. A close family member attended A&E with neurological symptoms, was discharged, and later suffered a stroke caused by a brain tumour. Replay is not suggesting AI would have prevented that outcome. Instead, it asks a different question:

If AI is involved in clinical decision-making, how can we reconstruct exactly what information was available, what recommendations were made, and why?

Replay explores that future.

What it does

Replay is a proof-of-concept AI governance platform set in 2035, where autonomous Clinical Investigation Agents assist emergency clinicians by:

  • reviewing patient history
  • analysing observations
  • retrieving current external clinical guidance
  • generating a structured clinical recommendation

The AI never replaces the clinician.

Instead, it acts as an intelligent investigation assistant.

Every investigation produces a structured AI Clinical Decision that can later be replayed, inspected and audited.

The project demonstrates both a successful emergency stroke investigation and a second, more ambiguous neurological presentation that later becomes the subject of a governance review. Rather than assigning blame, Replay reconstructs the investigation to help clinicians, hospitals and AI developers understand how the decision was reached and how future systems can improve.

How we built it

Replay was built as a modern full-stack application using:

  • Next.js
  • TypeScript
  • Tailwind CSS
  • shadcn/ui

The AI Investigation Agent orchestrates multiple tools during an investigation.

Patient context is combined with:

  • presenting symptoms
  • observations
  • ambulance handover
  • medical history
  • medications
  • external clinical guidance retrieved through Tavily

This evidence is passed to Google Gemini, which produces a structured clinical analysis including:

  • clinical assessment
  • supporting evidence
  • differential diagnoses
  • recommendation summary

Rather than exposing hidden model reasoning, Replay presents an observable, structured analysis based only on available evidence.

The resulting decision artefact becomes the centrepiece of the investigation and is designed to be replayed later as part of an auditable timeline.

Challenges

The biggest challenge wasn't building another AI demo.

It was designing a product that felt believable in a clinical environment.

Throughout development we continuously refined the workflow to ensure the AI remained an assistant rather than an autonomous decision maker.

We also wanted Replay to tell a compelling story rather than simply showcase technology. The final experience intentionally demonstrates both a successful investigation and a later governance review, highlighting that healthcare decisions are often made under pressure with incomplete information.

Another challenge was balancing technical transparency with responsible AI design. We wanted investigators to inspect prompts, retrieved evidence and AI outputs without implying access to hidden chain-of-thought or exposing implementation details to clinicians during normal workflows.

Known Limitations

Due to the time constraints of the hackathon, a couple of planned features remain unimplemented:

ClickHouse integration – The application was designed with ClickHouse-backed audit logging in mind, but the database integration was not completed before submission. Replay Execution Environment (REE) – A full replay engine was planned to allow deterministic replay of AI decisions. Instead, the project includes a mocked audit timeline that demonstrates how prompts, model outputs, and clinical reasoning would be captured and replayed in the final implementation.

These features are part of the planned roadmap and the current implementation was structured to make them straightforward to integrate after the hackathon.

What we learned

This project reinforced that trustworthy AI is about far more than model accuracy.

Building effective AI systems also requires:

  • transparency
  • observability
  • reproducibility
  • human oversight
  • clear governance

We also explored how modern AI applications can be built through orchestration rather than relying on a single model.

Replay combines external knowledge retrieval, structured reasoning and replayable investigation history to demonstrate how AI systems can become easier to understand, audit and continuously improve.

Ultimately, Replay isn't about proving that AI is always right.

It's about ensuring that when difficult decisions happen, we have the tools to understand them, learn from them and build better systems for the future.

Built With

Share this project:

Updates