main home page

Inspiration

Health and supplement claims spread quickly on TikTok and YouTube, but checking them often means reading product labels and dense research papers. We built MedBot to make that process faster and easier to understand.

What It Does

MedBot compares a video's claim with the product label and relevant scientific evidence. It highlights what is supported, misleading, or uncertain, then provides a short summary and animated explanation with sources.

How We Built It

We built MedBot with React and TypeScript on the frontend and Python, FastAPI, and Celery on the backend. The mobile-friendly interface lets users submit a short video, follow its progress, and explore the findings, sources, and limitations.

OpenAI transcribes the audio and extracts up to three spoken medical claims. We retrieve research through Europe PMC and supplemental health information from MedlinePlus, clearly distinguishing full-text papers, abstracts, and health summaries. Elasticsearch combines keyword and semantic search to find relevant passages. Those passages inform a structured assessment, with citations validated against the stored source text.

MedBot then turns one assessed claim into a short, captioned explanation. OpenAI generates the narration, local speech alignment synchronizes the captions, and FFmpeg assembles the video. PostgreSQL stores findings and progress, while Redis and Celery coordinate background work and support retries.

Sentry helps us understand failures and performance across the frontend, API, and workers. Error tracking, structured logs, tracing, and profiling help pinpoint failed requests and slow processing stages. Error-triggered Session Replay provides frontend debugging context with text and inputs masked and media blocked. Health check-ins help monitor availability, while sensitive transcripts, evidence, and model payloads are excluded from our application telemetry.

Challenges We Faced

The hardest challenge was simplifying complex evidence without overstating what it proves. Product formulations, dosages, study populations, and outcomes may differ, so MedBot must preserve uncertainty and clearly show where its conclusions come from.

What We Learned

We learned that responsible fact-checking requires more than asking an AI whether something is true. Strong explanations require transparent sources, structured comparisons, careful wording, and human review.