Example - HUVEC-18 cell profile image scaled 16x using our novel architecture
Early brainstorming of the model architecutre (left: variational autoencoder, right: convolutional neural network (discriminator)
Part 1: Encoder
Part 2: Decoder
We ran into some issues keeping the detail of image without blur when scaling, at first!
With most of our teammates passionate in research and academia, (Biotech, Astrophysics, Physics) we all saw a common issue amongst these fields being that traditional imaging techniques often impair the quality of research. This drove us to build SCALR, which is a deep learning instrument designed to improve image resolution while staying true to the content within the frame.
What it does
SCALR is a medical image processing tool aiming to improve resolution in cellular imaging by leveraging a generator autoencoder trained against a discriminator in order to generate highly detailed and sharp images with 16x the pixel density (4x the width and height). The target demographic is to help researchers in the drug discovery industry to better image cellular structures. This is done by extending an architecture developed by Nvidia, called SRRES-NET [https://arxiv.org/pdf/1609.04802.pdf], by turning the RES-NET into an autoencoder to force the model to encode high dimensionality features allowing for sharp image reconstruction.
How we built it
We began the processes by building the Super-Res Generator by prototyping the model in Python Notebook. We then migrated the prototype model into a GCP VM to train the very large model on a far larger dataset. Then the model was fined tuned, saved and integrated into a flask app to serve the model's features to potential clients displayed using ReactJS. This entire process involved various full-stack web dev and machine learning skills.
Challenges we ran into
During our process, we struggled to configure and maintain a python environment across all 3 systems causing major bottlenecks in our production cycle. We also struggled to integrate and connect react with the Flask application as the React sent files unreadable by flask. Despite these bottlenecks, we eventually overcame said challenges and delivered a polished app.
Accomplishments that we're proud of
We're proud that we brainstormed a novel method in training GAN's specifically relating to medical images. This technique involved training the generator against targets then injecting that model GAN which is trained with the feedback from the discriminator. This helped produce images with more clarity and sharpness with our overall results.
What we learned
We learned how important it is to maintain a clean, and well-documented codebase. None of us had a lot of experience collaborating on code, and so we ended up wasting some time trying to understand each others' code and how to make it work together. This took up some extra time but we think the learning experience was totally worth it.
What's next for Scalr
We would love to talk to companies and researchers working on cell-profiling or other biomedical images such as Novartis. To our knowledge, the architecture we developed is unique and we're sure people who are really knowledgeable in the field (ML & cell-profiling, for example) could give us a ton of advice on how to improve it.