GapFinder

An autonomous agent that maps and pulls scientific literature daily for what a research field has actually studied, and alerts you only when a verified gap gets filled based on your parameters.

Comment

Inspiration

What it does## Inspiration

I'm a cardiovascular postdoc. I study HFpEF, heart failure with preserved ejection fraction, and like every scientist I lose time trying to search hundreds of published papers, searching for the missing gap or experiment that can contribute to the success of overall science and the lab im in. The literature never stops moving, and the tools we have all do one of two things.... Some wait for you to ask a question and hand back a report and others email you when a keyword matches. Neither of them remembers anything between sessions and isnt persistent. Nothing keeps track of what a field already knows, so nothing can tell you when that knowledge changes and build upon itself as a interconnecting network with memory.

What I actually wanted was harder than searching. When I plan an experiment, the useful question isn't what's been published.... It's what hasn't. Which combination of animal model and measurement has nobody reported yet? Every tool that claims to find research gaps infers them from text similarity, on demand, when you prompt it. I wanted something that computes them, proves them with a number, and then watches for them to close.

What it does

GapFinder reads one narrow automated literature field, animal models of HFpEF for example, and builds a coverage map of species crossed with disease model crossed with outcome. Every square in that map sits in one of four states. It either holds evidence, meaning some number of papers measured that outcome there, or it's been checked with nothing reported, which is a verified gap. Or there aren't enough papers yet to say anything either way, in which case it says so instead of guessing. Or it's a combination that can't exist. ZSF1 is a rat strain, so a ZSF1 mouse square can never be filled, and it's never called a gap.

Every claim comes with its denominator. The system doesn't say a topic is under-studied. It says zero of five screened papers measured this, and here are the five papers.

Then it runs on a schedule. Each cycle it pulls its previous map out of storage, reads newly published papers, rebuilds the map, and compares it against yesterday's. It only alerts when a square changes state. A verified gap receives its first paper. A cell that held nothing but disease-description studies gains its first treatment study. Two papers that asked the same question the same way turn out to disagree. On most days it says nothing at all, and that's deliberate, because a monitoring tool you learn to ignore has already failed.

How I built it

Papers come from OpenAlex for discovery and PubMed for MeSH terms and abstracts. The corpus query is frozen in a config file so anyone can reproduce it. That pull returned 1,532 works, of which 368 passed the filter for primary animal research with a readable abstract.

Extraction runs on Gemini 3.5 Flash-Lite through the Google GenAI SDK, using a Pydantic response schema and a strict rule of verbatim or null. For each paper it pulls out the disease model and its components, any treatments tested, every outcome with its measure and direction and what that result was compared against, the proposed mechanisms, gene and protein names, and the study's time points.

The grid itself is deterministic Python with no model involved. Species comes straight from PubMed's own MeSH tags rather than from anything the model inferred. Disease models and outcome measures get canonicalized through mapping tables I can edit by hand.

The diff engine is the actual product. It compares two snapshots of the map, classifies every state transition, ranks them by severity, and returns an explicit "no material change" when nothing happened. It has nine self-tests.

For deployment, a Cloud Run job runs the daily cycle, triggered by Cloud Scheduler at seven in the morning Eastern. State lives in Cloud Storage with timestamped history so every past map is recoverable. API keys come from Secret Manager and never touch the container image. The dashboard is a separate Cloud Run service running under its own service account with read-only access to the bucket, so it physically cannot write to the state it displays.

For validation, the corpus is frozen at the end of August 2025, and a held-out set of genuinely later publications gets replayed as if newly published. Real papers, real publication dates, and I disclose that it's a replay rather than pretending otherwise.

Challenges I ran into

The input was broken before the model ever saw it. OpenAlex stores abstracts as an inverted index, and reconstructing them silently drops text from the middle. Twenty-six percent of my corpus was missing more than ten percent of its words, and one paper was missing sixty-seven percent. What made that dangerous wasn't the missing text, it was what the model did with it. Damaged input produced confident, well-formed, wrong records. One paper came back listing the animal's symptoms as the methods used to cause the disease. Not a blank record, a plausible one. My fill rates looked healthy the entire time this was happening. I only found it by chasing a single unexpected null. Switching the abstract source to PubMed's own text repaired a quarter of the corpus at zero additional cost.

My own contradiction detector was wrong three separate times, and I caught all three by reading the papers rather than by looking at any metric. It flagged an NLRP3 inhibitor improving cardiac hypertrophy against iron deficiency worsening it, and called that a disagreement. It isn't. A drug helping and a disease-worsening hurting are both doing exactly what you'd expect. My first fix split papers by whether they had treatment arms, and that failed too, because a tool compound sitting in a sub-experiment made a characterization paper look interventional. The real fix was a schema change. Now every outcome records what it was compared against, whether that's treated versus untreated, diseased versus healthy, or one subgroup versus another. A contradiction requires the same model, the same canonical measure, the same comparator, and for treatment results, the same drug. All three false positives cleared.

That precision cost me recall, and clustering bought it back. After the comparator fix, contradictions collapsed from fifty-seven down to two, because "diastolic dysfunction," "diastolic function," and "E/e′ ratio" were being treated as three separate concepts. Clustering 1,857 verbatim measures into 108 canonical ones restored the sensitivity without letting the false positives back in.

One deploy returned a healthy status on every endpoint with the feature completely unwired. A failed patch had landed the styling and the markup but not the JavaScript behind it, so there were visible controls attached to nothing, and every health check passed. Since then I verify that the content actually reached the server, not just that the server answered.

What I learned

A fill rate is not an accuracy metric. Every number I tracked for days measured how often the model answered, never whether it was right. The damaged-abstract bug and all three false contradictions were completely invisible to those metrics. They were found by a domain expert reading the output, which is to say by me, reading papers.

Saying "I don't know" has to be a first-class state. If you only allow two states, evidence or gap, then every blank square becomes a discovery. The third state, not enough papers yet, and the fourth, impossible combination, are what make a gap claim something you can actually check. The impossible-combination state paid for itself on its first run by catching a real bug, where papers tagged with multiple species were being filed under the wrong animal.

And silence is a feature you have to build on purpose. One cycle produced thirty-one raw changes and two alerts. Everything difficult about this project lived in the filtering, not in the finding.

What's next

Per-outcome arm attribution, so a tool compound in a sub-experiment stops looking like the treatment under test. Wiring up the live ingest path so it runs against genuinely new publications instead of a held-out replay. Watch lists for a specific gene or protein, alerting not when it gets mentioned but when it shows up in a part of the map where it never has before. And a config swap to a second field, to show the same engine works outside my own. Deeper persistent memory across the paper network, so the connections between papers carry forward between runs rather than being recomputed each time, and AI-assisted gap proposals that suggest which of the open gaps are actually worth an experiment.

Built With

Share this project:

Updates

Submission history