Shapes-n-Sizes

Dashcam footage with outline of objects + their node breakdown and shapes
Dashcam footage of what the GNN captured/shaped around

Inspiration

Self-driving futuristic cars. A lot of us in the Bay Area have been in Waymo cars or Teslas who are pushing technology further and further. We thought it was a super useful idea as you wouldn't need to drive yourself. However, we see that many accidents still happen with these autonomous cars that still run into various issues.

What it does

We use Graph Neural Networks to act in addition to what autonomous vehicles utilize. These GNNs help use calculate objects and identify them without appearance, texture, or colors. Instead, we use math, graphs, nodes, and edges to help identify objects. It then cross-checks back to the appearance AI from autonomous driving data to help identify issues, cross-check, and figure false-positive/negatives, etc.

How we built it

We used the ASUS Ascent GX10 to train/test open-sourced dash-cam data from BDD100K along with comma2k19 footage (MIT license) to later audit. Here we uses GNNs on the data to create a whole new testing batch of data being looked at just by nodes. The approach got rid of previous discrepancies and proved to catch issues that were not seen before.

Challenges we ran into

Training data since the more data we trained and tested on the GX10 would mean that we would have a better understand and results. The 24 hour time constraint proved to be a difficult time task to manage.

Accomplishments that we're proud of

We were able to produce a result and demo to share. Starting from the first few hours of the Hackathon, we had to pivot and rethink our idea multiple times, taking up a chunk of our day. I think this proved to push us to learn, adapt, and go through the process of producing a fully-built result.

What we learned

We learned how to create ideas and challenge our thinking in new ways. Failure did not mean giving up, but rather asking questions and gaining new perspectives. The various workshops, mentors, and other hackathon participants pushed us to create something influential.

What's next for Shapes-n-Sizes

Pushing more data into testing/training to create a history of results that benefit pre-existing companies like Waymo, Tesla, or other autonomous vehicle companies that lack the addition of shapes/graphs that makes this approach unique. Even then, we can still move forward from just vehicles but AI/computer vision for various other use cases.