AutoScope: Adaptive Malaria Microscopy

AutoScope turns a smartphone and a standard microscope into an adaptive malaria screening tool. It counts parasites automatically and adjusts how much of a slide to read based on the evidence.

Comment

Inspiration

Malaria remains one of the world's deadliest infectious diseases, causing more than 600,000 deaths in 2024. While rapid diagnostic tests are widely used, they cannot measure parasite density, which is essential for monitoring disease severity and drug resistance.

In many regions, microscopy is still the gold standard. The problem is that reading a slide is slow and labor-intensive. A technician may spend up to thirty minutes counting parasites and white blood cells before reaching a conclusion, and in understaffed clinics those counts are often skipped entirely.

As two researchers from Cameroon, we have seen how limited resources affect access to reliable diagnostics. Most district hospitals already have microscopes and trained technicians. What they usually do not have are expensive slide scanners or automated laboratory systems. We wanted to build a solution around the equipment that is already available.

What it does

AutoScope uses a smartphone attached to a microscope eyepiece to capture images directly from a blood smear.

The images are processed through an OpenCV 5 pipeline running on AWS Graviton4. The system corrects lighting and staining variations, assesses image quality, detects white blood cells, identifies parasite candidates, and estimates parasite density together with a confidence interval.

An AI agent then decides how much evidence is needed before reaching a conclusion. Depending on the situation, it can request additional fields, ignore poor-quality images, ask for refocusing, or send the slide to a human reviewer.

Importantly, AutoScope never makes a medical diagnosis. The agent operates within predefined safety rules, while limits on runtime, cost, and workflow decisions remain controlled by the surrounding system.

The result is a reading process that adapts to each slide. Slides with high parasite density may require only a few fields, while more difficult cases receive additional attention before a recommendation is made.

How we built it

The computer vision pipeline is built entirely with OpenCV 5 and OpenCV DNN. Our parasite classifier was trained in PyTorch, exported to ONNX, and deployed through OpenCV DNN, allowing the same model to run efficiently on both laptops and AWS Graviton instances.

The cloud infrastructure consists of 61 Terraform-managed resources. AWS services including S3, CloudFront, API Gateway, Lambda, Step Functions, Batch, DynamoDB, and Bedrock work together to orchestrate slide analysis and agent decisions.

One design choice we are particularly proud of is reproducibility. Every agent decision is cached together with the model, prompt, tool definition, and observation that produced it. More than 15,000 model calls are included in the repository, allowing researchers to replay experiments locally without access to AWS or additional costs.

Challenges we ran into

One of the biggest surprises came from performance optimization.

An OpenCV Marketplace image that performed well on standard benchmarks was almost four times slower than the regular OpenCV wheel when running our malaria pipeline. After investigating the issue, we discovered that part of the morphology stage was only using a single CPU core instead of all available cores.

We implemented a simple parallelization strategy that split images across multiple cores. This brought throughput back in line with the standard OpenCV build while producing identical parasite and white blood cell counts. We reported the issue to the OpenCV team, where it was reviewed and assigned for further investigation.

Another challenge involved the agent itself. Many models simply refused to stop reading a slide. Some continued requesting new fields until no more images were available. This taught us that being able to call a tool does not mean a model knows when to stop using it.

Accomplishments that we're proud of

One result surprised us.

After benchmarking eighteen different stopping strategies, we found that a simple fixed policy that reads eight fields performs almost as well as the adaptive agent in overall accuracy.

Instead of hiding that result, we reported it exactly as we observed it.

What the agent contributes is flexibility. It spends less effort on easy slides and more effort on difficult ones. The densest samples often require only a few fields, while sparse samples receive additional analysis before a decision is made.

Beyond that, we achieved an F1 score of 0.79 on held-out patients, detected 49 of 50 infected slides, reached a detection limit of 500 parasites per microlitre, and processed up to 547 slides per hour on a single AWS instance. We also made the system publicly accessible through a live demo with no login required.

What we learned

Benchmark numbers do not always predict real-world performance.

A system that appeared faster on isolated functions turned out to be significantly slower in our complete pipeline because of threading behavior. That experience reinforced the importance of testing complete workflows rather than relying solely on benchmark tables.

We also learned that model size is a poor predictor of agent behavior. Some of the largest models struggled to decide when enough evidence had been collected, while smaller models were often more willing to finish the task.

Most importantly, we learned the value of designing experiments that allow unexpected outcomes. The fact that a simple fixed policy competed with the agent became one of the most important insights of the entire project.

What's next for AutoScope: Adaptive Malaria Microscopy

Our next focus is moving beyond simulation and testing with real microscope hardware. Today, refocusing is performed virtually using captured focus planes, while physical microscope stage control remains future work.

We also plan to improve performance on Plasmodium vivax. While the system performs well on P. falciparum, accuracy drops significantly on P. vivax, making human review essential in regions where that species is common.

Finally, we are exploring confidence-weighted parasite counting. Early experiments suggest that using detection scores directly can reduce density estimation error without requiring additional training or major architectural changes.

Built With

Share this project:

Updates

Submission history