On a photography trip to the desert, Dennis broke the autofocus feature on his camera and while struggling to capture in-focus pictures, he realized that the focus ring does an excellent job of sweeping through the environment and providing a third dimension to images through depth. When he started to realize this observation and empower all kinds of users to experience the magic of 3D photography.
What it does
Our iOS app, when prompted, will take a series of pictures spanning through a known range of focused distance. This image set is then processed through our algorithms to determine the depth composition of the surrounding environment. Post-computation, the depth is combined with the original images to add another dimension to the original images. The potential of this technology is far beyond what we are able to realize in a limited amount of time.
How we built it
- Swift-powered iOS application
- Node.js and Express-based server app running on the Google Compute Engine
- core algorithms written in MATLAB, Python, and C++
- OpenCV to implement the designs
- WebGL to visualize the 3D images
Challenges we ran into
This project was a complete leap of faith, and the algorithm was developed from scratch through a combination of experimentation and research. As a computationally-heavy program at its core, the resource-intensive algorithms were difficult to optimize and even run on our own devices. None of us had any experience in Swift either, which presented to us a steep learning curve, especially given the limited timeframe and relatively complex system architecture.
Accomplishments that we're proud of
We're proud that we came up with a novel technique for 3D image acquisition on a standard single lens camera and that we were able to design and fully implement a working image processor. Furthermore, we worked as a cohesive, multidisciplinary team featuring electrical engineers, a computer scientist, and a mathematician, and we were able to stitch together very separate subsystems and make the final product work while managing to significantly optimize the backend algorithms in the short time from ideation to design, development, integration, and fine-tuning.
What we learned
- building an iOS app with Swift
- building a server app with the MEAN stack
- system design between mobile apps, servers, and SaaS cloud platforms
- file transfer via web protocols
- image processing
- multithreading, parallel processing
What's next for FociMap
We hope to further finetune our algorithms to be even more accurate, to optimize performance to the point where our projects can be quickly run as a standalone application on a mobile client, to build on and possibly formalize our work for academic publication, and to develop an open source library for implementation on more devices. .