As high school students with mediocre smartphones and thus, terrible smartphone cameras, we wanted to find a cure for this terrible blight. With previous experience writing articles and working on projects related to deep learning, we turned towards our trusted friend, TensorFlow, to help us fulfill our mission. Instead of emptying our wallets for a brand new Pixel 3 to satisfy our amateur photography desires, we searched for a solution with neural networks and we wanted to share our findings.
What it does
NeuraScale, the fancy name we gave our project, is basically a Super Resolution Generative Adversarial Network (SRGAN) with the purpose of upscaling image resolutions by a factor of two using deep learning. This way, a picture which initially appears pixellated and/or blurry can be modified so that the features are quite more distinguishable.
How we built it
We used TensorFlow 2.0 as the API for creating and training our SRGAN. The model was built with Keras and trained on the MS COCO Dataset. Numpy, Matplotlib, and several other libraries were used as well to allow for proper image preprocessing, as different image sizes need to be modified in order to be properly evaluated by the network.
Challenges we ran into
Since most neural networks require a fixed input/output size, figuring out the image preprocessing was a difficult part of our project, as we ran into many bugs and sighed in frustration multiple times. We found a way to split an image into several regular pieces to be fed into the network and then stitch the output together to end up with a proper, upscaled image.
Accomplishments that we're proud of
- Consuming only two Red Bulls each while working on our project.
- Buying way too many instant noodles.
- Using TensorFlow 2.0 to create a SRGAN model that can upscale images by a factor of two.
- Making a video for our project on way too little sleep.
- Totally NOT leaving everything to the last minute.
What we learned
This project gave us a chance to apply our experience with deep learning and TensorFlow to try and make something cool. We learned a lot from working with the new TF 2.0 API and about using Convolutional Neural Networks and Generative Adversarial Networks for feature extraction and image upscaling. Also, throughout the duration of the project, we played around some more with TensorFlow Lite and TensorFlow.js.
What's next for NeuraScale
Getting the recommended amount of sleep.
Just kidding. Our next steps for NeuraScale include smoothing out some of the rough edges in our code, converting the model to use it in a web app with TF.js or as a native app with TF Lite, and possibly retraining the model to output with colour enhancements.
Overall, this was an incredible learning opportunity and fun experience and we're stoked to see the other cool submissions! Make sure to go check out our github for more info, images, and usage details!