Inspiration

We saw a very clear opportunity in this hackathon to enter a field that was totally unknown for us, Computer Vision and Neural Networks. Being constantly out of our comfort zone is something that really attracts us and makes us feel constantly growing, which is why we decided to carry out this project without any member having any experience related with the field.

What it does

r3stD.ai generates from a video, a 3d model of a real estate space. This model includes an analysis of the property thanks to the AI ​​of restb.ai. We can summarize what r3stD.ai does in four big blocks:

  1. Process a video of the user, breaking it down into images.
  2. Reconstruct a 3D model from these images using the use of COLMAP technology and the NeRF algorithm.
  3. Annotations are obtained for the different elements of the model using the AI ​​of restb.ai.

Finally, r3stD.ai allows users to navigate the resulting model with annotations directly from the website, via Sketchfab.

How we built it

At first, we started by testing all the technologies and APIs separately. Shortly after, we began a process of integrating them. Starting with the development of the infrastructure (Backend and necessary Hardware), going through the development of a simple but useful user interface to finally integrate our system with the restb.ai API.

Challenges we ran into

  • Lack of time to find good training parameters for NeRF.
  • Incompatibility of the python open3D library on Windows.
  • Incompatibility of the graphics card GTX 1660 with NeRF.
  • Rendering of images of the 3D models with enough quality to obtain reasonable Restb.ai annotations.

Accomplishments that we're proud of

  • Successful integration of many technologies of very different natures.
  • Obtaining tangible results.
  • Successfully facing the challenges mentioned above and immersing ourselves in a totally unknown field.

What we learned

  • The most advanced algorithms and technologies for reconstruction of environments.
  • Understand the complexity behind integrating many totally different systems in a single environment.

What's next for r3stD.ai (restb.ai 3D)

  • Integrate the latent space of NeRF with the predictions of restb.ai.
  • User concurrence.
  • Greater infrastructure.
  • Batched/Queued execution of NeRF trainings.

Built With

Share this project:

Updates