Inspiration
This project was interesting because it provides an opportunity to combine AI with hardware in the real world for the mission. It also tests resiliency for this autonomous system in an unfamiliar environment.
What it does
You select a real world target area, our system fetche real time satelite imagery and flies 4 drones from commmand center to that target. To simulate blockers, adversary places jammers that drones are not aware of and hence they lose connection back to base, so they must now forma swarm mesh network connection, and communicate of where they are, what theyre seeing, and do healthchecks in heartbeat pattern.
We are also using YOLO computer vision for detecting, classifying objects in the satelite image.
As part of this simulation user has an option to take one or more drones down, observe the connectivity, and see how drone form plan to complete searching of target area and complete the mission.
How we built it
Our Tech Stack Layer Tech Frontend React, TypeScript, Vite Backend Python, FastAPI, Uvicorn Detection YOLOv8-OBB (Ultralytics), OpenCV Simulation NetworkX, NumPy Deployment Docker, Railway Mapbox API for satelite images
Challenges we ran into
hardware constraints, satelite image not rendering, drones not simulating oproperly, issues with computer vision, auto-navigation, etc.
Accomplishments that we're proud of
we have a good base working demo to start with in given time and learned about drone simulation stuff
What we learned
how we can design simulation softwares, things to considers, different technologiues to use etc.
What's next for Resilient Drone Swarm Search
Make it really production ready, utilize onboard LLM inference via LLama, have multi-auto navigation algorithms, and solid mesh networking, map the terrain out, have offline, fine tune model to search for items etc.
Log in or sign up for Devpost to join the conversation.