SnapAR3D

Inspiration

We were inspired by the increasing demand for accessible 3D modeling tools and the incredible potential of augmented reality. Seeing how AR could guide users to capture high-quality photos, we envisioned a tool that bridges real-world imagery and accurate 3D reconstructions.

What it does

SnapAR3D uses AR to guide you in capturing optimal photos of an object, while precisely recording the camera’s positions and orientations relative to that object. This data is transformed into matrices and imported into NVIDIA Diffractive Reconstruction (nvdifrec) to generate high-quality 3D models.

How we built it

We built SnapAR3D by integrating ARKit to capture photos and sensor data, and COLMAP to format the camera parameters. A custom conversion pipeline transforms the captured data into NeRF-compatible transformation matrices, which nvdifrec uses for reconstruction. We also developed a lightweight web host for easy sharing of the resulting transforms.json file.

Challenges we ran into

One of our biggest challenges was fine tuning—adjusting the photo capture process and data transformation parameters to ensure that the final 3D model was as accurate and detailed as possible. We also had to address coordinate system discrepancies between ARKit and NeRF, and seamlessly integrate data from multiple sources.

Accomplishments that we're proud of

Successfully guiding users with AR to capture optimal images.
Training a model that is comparable in quality and much faster in training time

What we learned

We deepened our understanding in AR app development, gained valuable insights into how 3D models are built using 2D photos, and learned to always push through the challenges because it will be worth it in the end.

What's next for SnapAR3D

Next, we plan to improve the user interface for an even smoother capture experience, expand support for diverse objects and scenes, and fine tune the model further while enhancing real-time feedback during data capture.