SwarmSense: Connected Intelligence for Connected Vehicles 🚗🤖

Inspiration 🌟

The inspiration for SwarmSense came from observing how swarms in nature, like bees or ants, work together to achieve complex tasks. We wondered: what if we could apply this collective intelligence to vehicles on our roads? With the increasing number of sensors in modern vehicles and advancements in AI and communication technologies, we saw an opportunity to create a system that could dramatically improve road safety and efficiency.

Original Goals 🎯

Predictive traffic flow optimization 🚦
Multi-modal reinforcement learning sensor fusion (lidar, camera, audio) 📡📷🎤
Parking assistance 🅿️
Intelligent agent based control system for accessibility and sustainability regardless of environment 🌍

What it does 🛠️

SwarmSense is not just a harmonic alliteration

It is:

Hyper aware agents who can truly perceive their world through...
Utilizing lidar
Cameras that see more than just colors 📷
Seamless localization through integrated spatial memory with SLAM 🗺️
Zero occlusion navigation through audio enhanced cognition 🎤
Through proprioception of the self and collective through intrinsic communication. 🔄

How we built it 🏗️

ROS2 utilizing a distributed control system. 🤖
Audio integration. 🎤
Raspberry Pi/OpenCR mobile robots.
LiDAR and Stereoscopic Cameras for perception. 📡📷

Challenges we ran into 🚧

Issues Porting to ROS2
Microphone Integration 🎤
MatLab Quest 🧩
One of our members' laptops gave up right at the start of the hack. 💻

How we addressed them 🛠️

Pivoted (Twice!) plan of attack to demo different scenarios that our solution addresses.
Moved away from microphone and eyed alternative methods of securing and transferring data. 🔄
Realized inefficiencies in MatLab - ROS integration. 🧩
The member sacrificed all of their data for the good of the team. 💾

Accomplishments that we're proud of 🏆

Developed a novel multi-modal approach for swarm perception
Ported software to run on ROS2 to create drivable robots, albeit with some issues. 🤖
Successfully integrated audio processing for emergency vehicle localization. 🚑
Created a scalable architecture capable of handling many more vehicles. 🚗🚗🚗
Most of our members were unfamiliar with ROS2.
Overcame agonizing sense of defeat and pulled through. 💪
AI from data, to deployment
Jetson nano deployment.

What we learned 📚

The critical role of simulation in developing and testing autonomous driving technologies 🚗
ROS2 building and deployment.
The basics of autonomous vehicles. 🚗
Image processing data pipelines. 🖼️
How to overcome incredulous challenges. 💪
Using OpenCV in Python. 🐍
Signal processing techniques.
How to use the MATLAB mobile app. 📱
Using Stereo camera to analyze depth perception. 📷
Building drivers from source. 🛠️

What's next for SwarmSense 🚀

Expand testing to real-world environments with partner automotive companies. 🌍
Integrate with existing traffic management systems for broader impact. 🚦
Develop additional use cases, such as optimizing parking in urban environments. 🅿️
Explore applications beyond road vehicles, such as in drone swarms or smart city infrastructure. 🚁🏙️