Airgap

Inspiration

Drug launch teams have to keep up with a constant flow of papers, trial updates, FDA reports, press releases and news. Most of it doesn't matter, but the few items that do have to be caught the same day. Right now teams either read everything by hand or paste it into a cloud chatbot. Pasting it in shows which competitor you're watching and what you plan next. Soon the same work will involve field notes that name doctors and patients, and that text can't go to an outside AI tool. We wanted a tool that does this on one laptop, with proof that nothing left the machine.

What it does

Airgap reads public items about two lung cancer drugs, Tagrisso and Rybrevant + Lazcluze, from 7 kinds of source. For each item it says what type of signal it is and whether the team should act on it. It also scores 5 aspects from −2 to +2 (efficacy, safety, convenience, access & cost, and evidence quality), says who made the claim, and gives an exact quote from the source to back it up.

A local dashboard shows the items that need action and how the tone changes month by month. It also has a Live tab where you paste new text and the laptop labels it. Before every run, 13 checks have to pass, including that the network is off and the model files match their published hashes. Each run leaves a log of every connection and a hash of every file, so a compliance team can audit it.

How we built it

We first tested more than 15 ways of labeling with large models. The simplest one won: Claude Opus with only the official labeling guide. We used it as a teacher to label the public practice data, and it never saw the test set. Then we trained LoRA adapters on Qwen2.5-3B with about 1,100 of those labels, using MLX on a 16 GB MacBook.

Structured FDA and trial items follow fixed patterns, so simple rules label them instantly. For everything else, three adapter checkpoints vote on the labels, the base model sets the tone and who made the claim, and a guard drops any quote that isn't copied word for word from the source.

Challenges we ran into

We only had 60 labeled examples, and every trick we tuned on them got worse on unseen items. One version scored 99 on its training items and 76 on the holdout, so we kept a holdout we never looked at. Early on, the small model joined pieces of different sentences into quotes that weren't in the source. That's why we built the guard.

The fine-tuned model found the right signals and quotes but often got the tone and the speaker wrong. The base model got the tone and speaker right but paraphrased its quotes, so we combined them. Running too many training jobs at once crashed the laptop overnight. Proving the run was offline was also harder than expected: in a dry run, background apps like Chrome, Slack and Zoom opened 48 connections in 5 seconds. Now we close them, turn on airplane mode, and log every connection during the run.

Accomplishments that we're proud of

The 3B model went from 60.7 to 81.8 on the official scorer, and scored 78.6 on items it never trained on. The Opus teacher scores 89.7. Every quote it outputs is found word for word in its source. The run makes 0 outside connections and costs $0 per 100 items. We also kept an honest privacy log: we sent the 60 dev items to Claude once to measure a ceiling, and no test item ever left the laptop.

What we learned

With very little labeled data, the official guide beats clever tuning. Different models are good at different parts of the task, and a simple vote can combine them. A claim of "private" only means something if you can show the evidence.

What's next for Airgap

Next we want to run it on internal field notes and advisory board summaries, where privacy matters most. Swapping the labeling guide and retraining should make it work for any drug. We also want to teach it to say "nothing here." It almost never labels an item as no signal, which is now its biggest error. The teacher used that label on only 8% of items, so the student never learned it. Finally, we'd like to turn the proof folder into a one-click compliance report.