3D-ify

Inspiration

From building architecture and home decoration to virtual clothes shopping, 3D models have impacted the way we approach aspects in everyday life. Yet such visualization has existed primarily for those with specializations in 3D modeling. 3D-ify introduces a friendly way to create 3D objects with just a phone.

What it does

3D-ify allows you to model objects in three dimensions, right on your phone! Given a portrait mode image from the modern day phone camera, 3D-ify generates a mapping from this image data to an easy-to-manipulate 3D model. This allows for further merging of two existing depth models, such that images of the same object taken from different angles may be combined to create a more complete and useful representation of the original object. This idea has more recently been adopted in industry for niche purposes, though this will be the first 3D-modeler targeted for the general audience: you!

How we built it

The implementation can be broken down into two parts: (1) iOS 3D display and interaction and (2) algorithmic optimization to merge two images. First, an iOS application allows users to select a portrait mode image, from which a 3D interactive representation is created from a combination of given RGB and depth information. Upon merging images, options allow the user to set priors, including setting the orientation of one image with respect to another. Not only does this allow for user interaction, it reduces the search space of possible configurations dramatically. Next, the information is sent to the backend algorithm. Much of the work here is to translationally align the two models in a computationally efficient way, which is necessary considering w*h of an image is ~10^5-10^6. An efficient version of grid search, as well as bucketing and sampling reduction, allows us to achieve an approximation for the rotation and translation amounts that minimize loss (RGB and distance).

Where to go from here

We hope to greatly simplify the prototyping and creation process with 3D-ify. Not only can this technology be used as an easy way to place new objects in augmented reality, it can be used to create prototypes of real objects within industry.