COVID-19 has forced students like ourselves to adjust to an unprecedented online learning environment. The proximity and density of university classes is highly conducive to the spread of a deadly pandemic; virtual learning has already become, and will be, the norm for the foreseeable future. If the return of in-person instruction is nowhere in sight, our objective should then be to optimize our virtual learning experience as much as possible.
What it does
Our project transforms a video so that it can be reflected onto a Pepper's Cone, creating a lifelike three-dimensional display on top of your laptop/tablet screen. Our first Pepper's Cone was created from a used orange juice bottle - hence the namesake Orange Juice Hologram.
How we built it
Our application is hosted on Google App Engine and uses the Flask web application framework. Our frontend is written in HTML and CSS. Our backend is written in Python with the support of various libraries including MoviePy, OpenCV, PyTorch, Matplotlib, and PIL.
Challenges we ran into
Google App Engine - it was difficult to manage videos in the read-only environment for GAE until we discovered the existence of a tmp directory. We worked around this with signed urls as well which was a cool feature to interact with cloud bucket from python.
Accomplishments that we're proud of
- We created a prototype of a larger pepper's cone (for our own entertainment in the future!)
- Running website deployed to GCP
- Using state of the art deep learning models to perform image segmentation
- Automating video editing with dynamic tile positioning algorithm that we designed
- Webapp pipeline from start to finish for creating 3D hologram video
What we learned
- Utilizing computer vision to do image segmentation to perform graphic manipulation
- First time using flask (very nifty)
- Utilizing GCP (first time!) (pretty cool)
What's next for Orange Juice Hologram
- Less scuffed demo video (sorry!!)
- Faster and more robust pipeline for video creation online
- Live conversion for use for video conferencing or other live video settings