Space Goggles

lidar-visualization
lidar-detection-2
lidar-detection-1
front-end
website-point-cloud-visualization

Inspiration

We all thought that the ICDS challenge was very interesting and a new topic for all of us.

What it does

Takes in PCD (Point Cloud Data) and detects items/objects given the data by running through a trained model and is displayed on a web app, showing the point cloud visualization as well as boxes representing identified objects.

How we built it

We started with research into libraries, frameworks, models, different vision LLMs, and picked a tech stack we believed would be best suited for the task. We built out the front-end with React and Bootstrap, developed our backend using the Django web framework and numerous libraries to create visualizations, organize data, make API calls, etc.

Challenges we ran into

Early on we were unclear about the goal of the challenge and where to begin, so we started off on a slow foot. As we started getting into it, one member solely was able to find a viable library for detecting objects, but he was the only one that was able to get it running without errors. Many of us were having trouble keeping up with all the imports and requirements, causing many halts in development time.

Accomplishments that we're proud of

We took on a difficult and new challenge head-on, creating a functional website that seamlessly displays our data in an organized manner.

What we learned

We learned how staying up over 24 hours makes your code suffers, how frustrating errors can be, and how developing on all MacOS, Windows, and Linux can cause some problems. We also learned how to effectively split up work, to enjoy the big wins when the code compiles correctly, and how fast a product can come to life when even just students work together.

What's next for Space Goggles

In the future we will add compatibility and support for item detection for phone lidar, image, and video processing. The end goal would to be able to have a camera or lidar sensor on a pair of glasses or headwear that can help those that are blind or with poor vision to identify what is around them.