Throughout the corona virus, we have seen the effect that boredom has on people and the subsequent media they generate. Our goal was to pivot from traditional compression, and use AI based compression to encode photo media.
What it does
We have built a photo storage app. When users submit a photo, we use BiCubic interpolation to downsample the image by a factor of 4 on each dimension, resulting in a total decrease in data of 1600%.
We store these latent downsampled states on a server, and when a user requests there photo, we upsample the photo using AI for super resolution. This returns the photo to its original state for the user.
How We built it
Our three-person team split into three different parts. One was responsible for the AI/backends scripts, one was responsible for the AI and the server architecture, and one use responsible for UI/UX/Rounding out.
To combat the problem of low quality, we applied the SRGAN (Super Resolution Generative Adversarial Network) model [https://arxiv.org/pdf/1609.04802.pdf] algorithm to upscale the image. A generative adversarial network is a neural network where two separate models are trained in competition witch each-other. In our case, one model learned to upsample images and the other model learned to detect upsampled images. These two models compete against each-other, each making the other improve. We fine-tuned this algorithm through further training and wrote additional python scripts to process this data for ready use by a flask backend. This flask backend then worked with a node server to deliver our speedy image-text/dm/ experience to an iPhone app. Our iPhone app also includes a thumbnail feature, where users are still viewing previous photos in low-quality thumbnail form until they click on the picture and access the high quality version.
Challenges I ran into
We ran into a lot of challenges with time. We originally planed to serve the SRGan locally through the users phone; however, integration with Swift Lite was a problem. This led to us being short on time in the rest of our work.
Accomplishments that I'm proud of
We are proud of the idea and tech that we have presented. The compression we present here we believe to be very promising as it can keep being pushed and pushed. While other image compression algorithms such as JPEG are hard coded and have relatively met their ceiling, our AI based compression is just beginning and will only improve along with the state of AI.
What's next for Limitless
We are very interested in the idea of using this compression to reduce bandwitdh when sending image media. We originally wanted the models to be present on the phones such that someone could send and receive compressed images and resample them. We are also very interested in applying this tech towards video streams.
Explaining our Code Structure
We have two gits (Serverside is backend is nicknamed FastImage) the UI/client is called Limitless.