DrugSignal

Pipeline
Technical Design

Inspiration

Antibiotic susceptibility testing can take one to three days, while clinicians must often begin treatment immediately. We built DrugSignal to explore whether an assembled bacterial genome could provide an earlier, evidence-backed prediction of which antibiotics are likely to fail—without pretending that a model can replace laboratory testing.

What it does

DrugSignal analyzes an assembled Staphylococcus aureus genome and predicts resistance for cefoxitin, ciprofloxacin, erythromycin, and tetracycline. It identifies resistance genes and mutations with AMRFinderPlus, combines them with interpretable machine-learning models, calibrates the probabilities, and reports one of three outcomes:

likely to fail
likely to work
no-call when the evidence is uncertain

Every failure prediction must be supported by evidence from the isolate, and every report includes a laboratory-confirmation disclaimer.

How we built it

The project is organized into three modules:

Genome Reader — runs AMRFinderPlus and converts detected genes, point mutations, and drug-class signals into binary features.
Predictor — applies a deterministic target gate and one L1-regularized logistic-regression model per antibiotic.
Decision Report — calibrates probabilities on a held-out split and presents the prediction, confidence, evidence category, and evaluation metrics.

To make the evaluation more realistic, genomes are split by genetic cluster rather than by random rows, preventing near-identical isolates from appearing in both training and test sets.

Challenges we ran into

One challenge was avoiding misleading confidence. A model trained on a resistance-heavy cohort could predict failure for a clean genome using only the training-set base rate. We added a coherence rule that downgrades unsupported failure predictions to no-call.

We also found compatibility issues with AMRFinderPlus v4 output headers and drug-class vocabulary. These issues could silently discard real resistance signals, so the parser now accepts known schema variants and raises an error for unknown ones.

Another challenge was representing uncertainty honestly. A no-call is not a high-confidence result, so we separated resistance probability from decision confidence and show confidence only when the model actually makes a call.

Accomplishments that we're proud of

We built an end-to-end pipeline that runs from FASTA input to an explainable resistance report. We are especially proud that:

predictions are tied to specific biological determinants where available;
cluster-level splitting reduces lineage leakage;
calibration is performed on data separate from the test set;
unsupported failure calls are explicitly refused;
no-call results prevent forced, overconfident decisions;
the system clearly separates known mechanisms from statistical associations.

What we learned

We learned that responsible machine learning is often about refusing to overstate what the data supports. Calibration, leakage prevention, schema validation, and transparent evidence categories matter as much as model accuracy.

We also learned that genomic prediction is not simply a classification problem. The system must communicate uncertainty, distinguish correlation from mechanism, and preserve human oversight—especially when the output could influence treatment decisions.

What's next for DrugSignal

Next, we want to expand validation across larger and more diverse clinical cohorts, add support for more organisms and antibiotics, and improve generalization testing on previously unseen lineages.

We also plan to improve the user experience, add richer report exports, expose more detailed model diagnostics, and strengthen integration with laboratory workflows. The project will remain defensive by design: a decision-support tool for studying existing resistance, never a system for designing or optimizing organisms.