ShutterSense

Find exactly how the pros took their shot

Comment

Inspiration

All of us have seen various photos that made us sit there and under "How did they take these photos?". We wanted to build an app that would help users understand how the professionals shot their photos and better there own learning.

What it does

PhotoSense takes in a user submitted photo and analyzes it to eventually output various settings on the camera necessary to emulate the photo. Specifically it returns the closest match it has for aperture, exposure time, ISO, shutter speed, focal length, metering mode, flash. Beyond that, we also integrated in a chatbot to help the user further tailor settings or brainstorm ideas.

How we built it

We followed the MVC framework: model, viewer, controller. The model was written in Python and consists of the analysis of the user-submitted photo and the closest picture the model has to the photo. The viewer was built using React and takes in user-submitted images to pass them to the controller. Speaking of the controller, we used Springboot as our framework to pass data from the front end to the model and back.

Challenges we ran into

Initially we wanted to use a MySQL backend. However, we quickly realized that this did not accomplish what we had initially hoped. We needed to store images, which a SQL backend simply could not do. We eventually switched to using Amazon S3 to store images which required a refactoring of our backend. We also had to update our training set to have more diverse anchor images to better tune the model because it fell short of validation expectations. We also tried and failed to integrate multiple LLMs until we found one that responded properly through our RAG chatbot. Finally creating a mutli-page app with proper state management (and reduced prop drilling) had its own challenges. Some of our largest difficulties were with getting each individual piece properly communicating with other sections with our project.

Accomplishments that we're proud of

A working clip model. A full stack app that combines React, Java, and Python. The app is multi-page, interactive, houses a working CLIP model, and Java Controllers managed by the Springboot framework. It was a gratifying moment to see everything finally come together at the end!

What we learned

Understand the usage and role of each piece of technology so that time isn't spent on areas that are not needed. We also gained a better understanding of the MVC framework and how a modular design can help speed up development.

What's next for PhotoSense

We want to add more camera settings and allow for finer granularity in the camera settings. There are many more permutations of camera settings that we would like to capture. There are also more pertinent variables that we would like to account for, such as the white balance and the angle of the camera. All of these is achievable with more data. In addition, the ability for users to help contribute to our training data by uploading photos with pertinent information would be critical for the long-term longevity of the project and to make our model more robust.

Share this project:

Updates