PharmaGuard

• 

  Research workspace for drug-safety questions, deterministic facts and traceable source evidence.

• 

  A live cited answer alongside reporting signals computed from the published FDA dataset.

• 

  Inspect versioned label excerpts, dataset coverage and limitations without leaving the investigation.

Inspiration

A drug-safety answer is only useful if someone can inspect the evidence behind it. Adverse-event reports, drug labels and published studies answer different questions, and a fluent summary can hide missing support. I built PharmaGuard to bring those sources into one research workspace and to make evidence coverage part of the release process.

What it does

PharmaGuard lets a researcher ask a drug-safety question, review reporting signals and inspect cited source passages. The initial investigation scope is semaglutide and pancreatitis. PostgreSQL calculates case counts, reporting odds ratios and proportional reporting ratios; the language model explains the retrieved evidence without calculating those metrics.

Answers link back to canonical evidence with source versions and dates. Historical cutoffs, unavailable data and insufficient evidence remain visible. This is a bounded research prototype for signal investigation and evidence review, not medical advice or proof of causality.

How I built it

The interface uses Next.js and TypeScript, with a server-side proxy to a private FastAPI backend. PostgreSQL / Cloud SQL holds authoritative facts and canonical text. Google Cloud Storage preserves raw artifacts and frozen evaluation snapshots. MedCPT encoders and a reranker retrieve evidence through a rebuildable Qdrant index, and Vertex AI supports structured, cited synthesis.

On Google Cloud, Cloud Run provides the public HTTPS entry point for the VM-backed release path on Compute Engine. Images are stored in Artifact Registry, configuration and credentials come from Secret Manager, and Terraform defines infrastructure. GitLab CI checks software, security and evidence quality before release progression.

The GitLab Duo workflow

The recorded demonstration follows a release candidate whose retrieval depth is set to top_k: 1. Its rag_eval gate catches missing required evidence. A repair issue gives Duo the failed report, a passing baseline and a tightly bounded scope on the existing candidate branch.

Duo investigates the failure, restores top_k: 5 in config/retrieval.json and pushes commit 074080c0. Candidate pipeline 2973012295 then passes. The small diff, evaluation results and Duo summary make the repair inspectable.

The same ten reviewed cases, approved snapshot, model revisions and thresholds stay fixed. Each case must achieve Recall@5 of at least 0.80, retrieve every critical evidence ID and preserve exact facts, eligible citations and temporal safety. These checks measure the reviewed benchmark; they do not prove unrestricted clinical accuracy.

This is a supervised workflow: Duo investigates and implements a bounded repair, while a human reviews the outcome, merges and approves release actions. A passing candidate pipeline and the post-merge main pipeline are separate pieces of evidence. Duo does not merge or promote production automatically.

Challenges

The difficult part was making an evidence failure visible even when ordinary software tests still pass. Retrieval depth can change the support available to an answer without breaking the API contract. Frozen, source-reviewed cases make that regression measurable.

Another challenge was keeping source dates, canonical citations and deterministic facts consistent across ingestion, retrieval and evaluation. A valid citation identifier alone does not prove that its passage supports a claim, so reviewed expectations and separate live judging supplement the deterministic gates.

What I learned

An agent needs a concrete failure, accessible evidence and a clear repair boundary. A green pipeline is meaningful only when the underlying tests and evidence stay intact. Human release decisions also need the actual diff and validation results, rather than a confident agent summary alone.

What is next

I want to expand the reviewed drug/event coverage, strengthen expert review of evidence and answer support, and make dataset freshness and evaluation limitations easier to inspect. The next improvements should preserve the same principle: traceable facts, measurable evidence quality and human-controlled releases.

Evidence

• GitLab project and source
• Retrieval repair merge request !15
• Passing candidate pipeline 2973012295
• Recorded demo

Built With

• cloud-run
• cloud-sql
• compute-engine
• docker
• fastapi
• gitlab
• gitlab-duo
• google-cloud
• medcpt
• next.js
• postgresql
• python
• qdrant
• rag
• secret-manager
• terraform
• typescript
• vertex-ai