Dark Sight

Dataset
How we made it
More samples

Inspiration

Existing Systems for night vision only perform image amplification, thereby introducing noise in the images, leading to blurry or unclear images that cause confusion. They also produce only black and white images with green filters. Another technology widely used is infrared-based thermal signature imaging widely known as thermal cameras. These cameras are heavily restricted by their field of view. Also, they are conditionally opaque to materials falling within a specific refractive index.

What it does

Our model changes the way night vision cameras work. We are building a system that can produce coloured images with very low distortion. We wish to reproduce bright images in dark environments while also operating in real-time.

How we built it

We collected a custom dataset with sets of 3 images for each object.

Low Exposure Image (extremely dark)
Thermal Reading
High Exposure Image(visible)

The High Exposure image acts as ground-truth for training the model, while the other two images are the training inputs. All three images are of the same object and have been taken simultaneously.

Then we preprocessed the data in the following manner :
We first matched the sizes of the thermal and input image to feed to the network.
Then, To improve the training of our model, we applied data augmentation techniques like randomcrop and randromflip on the input images simultaneously.
Cleaning of the dataset was achieved by the randomsampler function
Finally both images were concatenated and fed to the network.

Deep Learning Model - We use A UNet architecture which consists of an encoder and decoder. First, the encoder transforms the image into efficient encoding using convolution operations. The decoder uses this encoding to construct the output image along with the aid of skip connections from the encoder. The model learns to infuse information from thermal and regular images.

Challenges we ran into

Getting a custom dataset suitable for our needs
Real-time generation of images required a higher frame rate
GPU computations for training were very hefty

Accomplishments that we're proud of

Achieved amazing results with our deep-learning model in a short time.
The model is also computationally efficient and has a good real-time performance.

What we learned

We learnt the intricacies of a deep learning-based pipeline and how each component is essential for the efficiency and quality of the output. Image processing is a continuously growing field, so going through research papers led us to various advancements and techniques that we could understand and try to incorporate into our solution.

The most important thing that we learnt was that technology paves the way for tackling the biggest of problems and in spite of the pandemic, we could come together and work on this solution from the comfort of our homes.