MiniManpad

Making asymmetric drone warfare against dismounted troops symmetric - low cost mini manpad against drones.

Comment

Inspiration

Inspiration: Troops in Ukraine don't have effective, cost efficient and scalable ways of protecting themselves from the bombs of an incoming kamikaze drone or the eyes of an ISR quad.

What it does

We built the sensor system for a micro-MANPAD only reliant on optical sensors and running on cheap hardware. We built a program and deployed it on a Raspberry Pi 3 to lock onto and quickly track drones. It can do this at well over 60 FPS by detecting every it detects every 5 frames and then tracks the drone and estimates its future trajectory between those frames.

How we built it

  1. Collected a bunch of different datasets for drone identification and tracking.
  2. Trained a model (Detectron2go) from scratch on an H100 using the MS COCO dataset.
  3. Fine-tuned the model on drone videos with labeled bounding boxes.
  4. Implemented a tracking system using a Kalman filter.
  5. Implemented template matching using SIFT features in the bounding boxes for frames between the neural network detections to improve tracking and estimation.

Challenges we ran into

  • Datasets are a mess.
  • Dependency hell is real (RPI5, TPUs, py versions etc)
  • Models not available.

Accomplishments that we're proud of

  • Fully trained Detectron2go model from scratch running on a RPI5 at 25+ fps
  • Integrated Kalman filter estimation and tracking
  • multi-type detection using both efficient template matching and neural network detection for increased FPS.

What we learned

  • Prepared data is crucial for getting something out quickly, spending too much time on data for a hackathon is detrimental.
  • Awesome to work with ppl with very varied and different background.
  • Splitting the work early on to make sure everyone is clear on what to do.

What's next for MiniManpad

  • The detection algorithm is running at 27fps, faster than it needs to, since we can track between the detection frames. Therefore we want to trade some of that speed for the ability to take in higher resolution images and track drones at a longer distances.
  • We want to collect and label more realistic training data to make it applicable for the Ukrainan frontline
  • Then we want to integrate the detection system with a model rocket with steerable fins, to see the challenges with actually intercepting a quickly moving drone.

Built With

Share this project:

Updates