Unsay

An AI medication-safety agent that revisits its past answers when FDA recalls change, identifies affected patients, and generates personalized corrections for pharmacist approval.

Comment

Unsay — An AI Medication-Safety Agent That Can Correct Its Past

Inspiration

A medication recall is scoped to a lot number. But what happens when a system cannot connect that lot number to the specific patient who received it?

Hospitals and pharmacies face a difficult choice: warn everyone or risk leaving affected patients uninformed.

We identified a second problem in AI systems: stale memory. An AI agent might retrieve information that was correct when it was stored but is no longer true.

In healthcare, this is more than an inconvenience. An outdated medication-safety answer could have serious consequences.

These two problems inspired Unsay.

We wanted to build an AI agent that does not simply answer questions using current information but can also revisit its previous answers, identify which ones have become unsafe, and correct them.

What It Does

Unsay is an AI medication-safety agent powered by live openFDA recall and drug-labelling data.

Its defining capability is the ability to go back and correct previous answers when new safety information becomes available.

When the FDA escalates a recall, Unsay:

  1. Identifies the updated medication-safety information.
  2. Finds previously issued answers that relied on outdated claims.
  3. Re-evaluates each answer using the latest available evidence.
  4. Identifies the specific patients whose answers need correction.
  5. Generates personalized correction messages for pharmacist approval.

Hosted Demo

Our hosted demonstration includes twelve patients who received medication from one real recalled lot.

  • 12 patient records examined.
  • 9 previous answers reversed.
  • 3 answers left unchanged because those patients had already been advised to stop taking the medication.

This ensures that corrections are generated only when the underlying answer actually changes.

How We Built It

1. Bitemporal Memory with CockroachDB

CockroachDB is the foundation of Unsay's memory architecture.

Every safety claim maintains two timelines:

  • valid_from and valid_to: When a claim is true in the real world.
  • asserted_at and retracted_at: When the system believed that claim.

This distinction allows Unsay to understand the difference between when a medication became dangerous and when the system discovered that information.

Claims are never overwritten. Instead, the previous version is retracted and a new version is asserted within a transaction.

2. Answer Provenance

Every answer records the exact versions of claims it relied on.

The answer and its provenance are written in the same transaction. This means the system can identify which previous decisions depended on information that has since been retracted.

3. Evidence-Based Recall Sweeping

Unsay uses SQL queries to identify standing decisions that relied on outdated safety claims.

Rather than relying only on vector similarity, it tracks the exact versions of information used by previous answers.

This makes historical decision correction possible.

4. AI Reasoning and Retrieval

  • Claude Sonnet 4.5 on Amazon Bedrock: Handles reasoning and generates answers with explicit citations.
  • Amazon Titan V2: Generates embeddings for semantic retrieval.
  • CockroachDB distributed vector index: Supports retrieval while filtering out retracted claims.

5. Cloud Infrastructure

The hosted application runs on AWS Lambda and connects to CockroachDB Cloud.

We also developed multi-region resilience capabilities using a 9-node, 3-region CockroachDB cluster.

Technology Stack

  • Python
  • CockroachDB Cloud
  • Amazon Bedrock
  • Claude Sonnet 4.5
  • Amazon Titan V2
  • AWS Lambda
  • openFDA
  • FastAPI
  • Boto3
  • Docker
  • SQL
  • Vector Search

Challenges We Faced

1. Long-Term Memory and MVCC Limitations

Our initial approach used CockroachDB's AS OF SYSTEM TIME to replay historical database states.

However, this approach depends on the MVCC garbage-collection window. It cannot provide reliable long-term historical retention.

We moved to an application-level bitemporal schema, keeping MVCC time travel as a faster path within its available retention window.

2. Reliable Memory Supersession

We encountered multiple problems with AI-generated memory extraction.

The model sometimes returned conversational responses instead of structured JSON. It also generated different keys for the same attribute, preventing older facts from being superseded correctly.

We improved the extraction process and enforced consistent attribute keys in code.

3. Deployment Challenges

Deployment revealed issues that local testing had not caught:

  • Missing transitive dependencies in the Lambda deployment bundle.
  • AWS Lambda Function URL permission requirements.
  • Incorrect regional health-status reporting.

We addressed these issues through deployment-level debugging and infrastructure improvements.

4. Cost and Demo Reliability

Because the recall sweep modifies previous decisions, repeated demo visitors needed a fresh scenario.

We implemented automatic scenario restoration, response caching, request deduplication, and database-backed model-call limits.

Accomplishments We're Proud Of

Concurrency Testing

We tested the bitemporal memory architecture with 64 concurrent writers, with 16 writers racing on each of four claims.

Results:

  • Zero failures.
  • No gaps or duplicates in version chains.
  • Exactly one believed version per claim.

Public Demo Reliability

We implemented:

  • A limit of 200 model calls per day.
  • A limit of 40 calls per visitor per hour.
  • Cached opening answers.
  • Deduplication of identical questions.
  • Automatic demo scenario restoration.

Honest Benchmark Reporting

We evaluated Unsay on LongMemEval's hard split using the official GPT-4o judge and achieved 20.5%.

The benchmark exposed a retrieval limitation: many incorrect answers resulted from insufficient retrieval coverage.

We documented the result and its limitations rather than hiding the performance gap.

What We Learned

Our biggest lesson was that AI memory is not just about storing and retrieving information. It is about knowing when information was true, when it changed, and which decisions depended on it.

We also learned that deployment exposes problems that traditional testing cannot always identify.

Building Unsay helped us understand distributed databases, temporal data modeling, AI reasoning, vector retrieval, and reliable cloud deployment.

Most importantly, we learned that trustworthy AI must be capable of acknowledging when its previous answer is no longer correct.

What's Next for Unsay

Our next priorities include:

  1. Improving medication lot-number extraction using real pharmacy dispensing records.
  2. Reducing over-notification when recall information lacks specific lot numbers.
  3. Improving retrieval through adaptive top-K selection and recall-oriented reranking.
  4. Expanding the system's ability to identify and correct outdated safety information.

Our long-term vision is to make AI agents accountable for what they have previously told people.

Because in medication safety, giving the right answer today is not enough. An AI must also be willing to correct yesterday's answer.

Built With

Share this project:

Updates