We want to create a 3D point cloud mapping tool for robotics and autonomous driving applications using the novel computer vision technique visual SLAM.
What it does
We create 3D point cloud maps of physical environments using feature-based visual SLAM. Feature-based visual Simultaneous Localization and Mapping (SLAM) is a computer vision technique to perform location and mapping functions in an unknown environment by tracking and matching feature points between video frames. User can upload a video of an observer exploring a physical environment to create a 3D point cloud map of the environment, which is visualized in Three.js.
How I built it
We implemented visual SLAM in C++ with the help of OpenCV. The web app is built with Flask, Python, Three.js, SQLite, and GCP.
Challenges I ran into
Implementing visual SLAM algorithm from scratch is complicated and debugging C++ can be difficult. I had to implement a huge amount of functionality, like a generalized RANSAC discriminator and circular reference counting. The hardest part was creating the pose graph between frames.
Accomplishments that I'm proud of
Despite all the challenges we were able to create a web app that allows everyone to try out the visual SLAM technology through the web app. We are also proud of being able to implement visual SLAM algorithm from scratch.
What I learned
We learned about visual SLAM and how to build a web app around it.
What's next for 3D point cloud mapping vSLAM
There is so much to do, we could use KeyFrames for higher base lines, thread our RANSAC, perform radius searches better, etc. We would optimize our visual SLAM algorithm to make it run faster.