Room Reveal

Room Reveal Logo

Inspiration

The need for this project was inspired by the fact that currently, Purdue only provides a top-down floor plan of most dorm rooms without any pictures or videos. The floor plan leaves out important details such as the location of furnishings and actually usable space. We aim to provide a more immersive and informative way for students to explore their future living spaces before moving in. Room Reveal helps students explore living spaces in dorms or apartments around campus, helping them plan ahead for how to effectively use space, what to buy, or what accommodations they may need. This is especially useful for those with disabilities or accessibility needs, as they can get a better sense of the space before moving in. Different room configurations such as lofted beds, double vs triple occupancy within the same room type, and possibilities for furniture arrangement can be better visualized through our platform.

What it does

Room Reveal is a platform where students can explore 3D Gaussian Splat models of various living spaces around campus. Models are created when users upload a video of their rooms (after specifying their location and room type) that is transformed into a Gaussian Splat 3D model by our processing pipeline. From there, any user can explore the space interactively on our website to get a sense of potential future living spaces.

How it works

The process of creating a Gaussian Splat model from a video involves several steps:

Data Collection: The user uploads a video of their room, which can be taken with a phone camera. The video should capture the entire living space from multiple angles to ensure good coverage for 3D reconstruction.
COLMAP Processing: The video frames are extracted and processed with COLMAP, a structure-from-motion software, to extract camera poses and a sparse point cloud of the scene.
Gaussian Splatting: The extracted data from COLMAP is then used to train a Gaussian Splatting model using Nerfstudio, which creates a 3D representation of the scene in the form of Gaussian splats.
Visualization: The resulting Gaussian Splat model is then made available on the Room Reveal website, where users can interactively explore the 3D scene to get a better sense of the living space.

How we built it

Our project has two major parts, which we tackled concurrently:

Pipeline for video (or set of photos) -> Gaussian splat (.ply)
Interactive web interface to enable users to explore the 3D Gaussian splat scene

The Gaussian splatting pipeline uses a nerfstudio container, in which we run the COLMAP and train the Gaussian splatting model to accurately represent the space given the extracted images (with positional data from COLMAP). The final 3D model is exported as a .ply file, which is used as the input for the second stage of the project.

The web interface takes in a .ply file as input, and renders the 3D model with Three.js and SparkJS for interactive Gaussian splat visualization in the browser. We spent time tweaking the controls, movement speed, initialization point, and scene orientation until the user experience was truly intuitive.

Challenges we ran into

This project was definitely not without challenges. The Gaussian splatting pipeline development was particularly difficult because of the complicated setup and debugging required to run a locally-hosted GPU-enabled nerfstudio container for training the model. A good amount of time was spent finding and setting environment variables to circumvent permissions-related errors inside the container, or correctly mounting the filesystem to persist model cache and pipeline output files.

The web pipeline came with its own hiccups during development, from the model not rendering, to the model not rendering with color, to the model rendering upside-down, etc. ;) Despite the issues, nothing beats the first time you see a fully rendered 3D model of something you captured yourself, with a Gaussian splatting model and pipeline you built yourself!

What we learned

We learned a lot about Gaussian splatting- how it works, what is required to build a pipeline around it, and how to display it. Implementing this project took us far beyond conceptual understanding, since we were implementing the actual pipeline and visualization ourselves. We learned the ins-and-outs of Docker, NerfStudio, Three.js, Spark.js, and other technologies that we used to create our project.