Dronautics

Revolutionizing drone control with wireless local detection and GPS tracking at a very low cost

Comment

Inspiration

Most drones are limited by a simple factor: the fact that you have to control them. Furthermore, the drones that have built-in sensing, I feel, are too expensive for regular consumers, and also not very practical for day-to-day usage.

I really wanted to make a system that treats the drone as a supporting actor on everything that you perform. Furthermore, I wanted to make it easily affordable, and implementable across any programmable drone line. The "Leash" drone system that I came up with, alongside some built hardware accessories, can help in achieving the accuracy required to transform this into a real product that can aid many different people in different professions, such as construction workers, firefighters, policemen or general consumers.

What it does

Dronautics is a wearable brace that flies a drone based on body language. Follow: the drone holds a set distance from you as you walk. Radius by button: Closer (White button) and Farther (Black button) on the brace move the leash 0.4 m per press, with a haptic buzz to confirm the action took place. Orbit: the drone circles you while its camera stays locked on you. Done by making the drone move around you in an arc while the camera locks the person in the center. Target lock: with several people in view, it follows only you. Lost and found: if you leave the frame, it looks where you were heading instead of freezing. If fully lost, the drone will start looking in a 360-degree radius, Phone control center: a live map with drone support, the camera view, modes, simulators, the radius, and an emergency stop, all on a nicely designed GUI.

How I built it

This project consists of four technological layers, each one talking to the next: the brace measures your location and angle, a base station relays this information with 30-byte packets, the laptop decides what to do with those packets, and the drone flies.

The brace: an ESP32 reads a BNO085 compass (heading, otherwise the degrees that a person is facing, initialized by the button "a"), two buttons (radius controllers), a DRV2605L haptic driver (feedback by using a vibration motor), and a u-blox MAX-M10S GPS. It broadcasts a 30-byte packet 20 times a second over ESP-NOW. The base station: an ESP32 on the laptop's USB turns each packet into a JSON line. Did this to prioritize safe, controlled handling over drones on general public areas. The brain (Python): YOLO on the GPU finds and locks onto you. Your box gives a bearing and a distance, the brace gives your facing, and the leash calculator places a target point beside you. The YOLO tracker makes sure to scan over your shirt color for further locking of your identity. The app: Expo + React Native + TypeScript, linked to the laptop over WebSocket's, designed in Figma first. The app hosts a simulation mode, specifically made for judges that want to utilize the drone in an educational, no-risk manner. AMD Cloud RIFE, an open-source frame-interpolation model, runs in PyTorch on ROCm on an AMD Radeon Pro W7900D. It turns the drone's ~15 fps into ~30 fps for the human watching. The drone's AI only ever sees real frames, to not confuse the data tracking. Two streams are visible to the person, to compare and contrast the given interpolated footage.

Challenges I ran into

  • I failed in reverse-engineering a toy drone. I originally wanted to make the drone in this project from an old model Snaptain. As I started to program inside of it, I realized that the streaming server for the drone was inaccessible, even if you sent tiny "heartbeats" inside of the drone. Because of this, I decided to go for a really simplistic model for this project and try my best to make the model as advanced as possible, even with a strict budget. Because of this, I landed on a TJI Tello Drone, with a 720p camera, 10 to 2-minute battery span, and no sensor usage.
  • I also did some rewrites that made the project worse. After the hackathon started, I had a version stacked on separate mathematical rules, each assuming something unchecked about the drone. I rebuilt it around one geometric idea, the target point, plus calibration steps that turn assumptions into measured numbers. Furthermore, every single tool in the brace is used. All of the calibrations and stability aides, alongside pathfinding data present in the drone, is due to usage of every single tool present in the brace.
  • The insane video lag also was a hard challenge. With the Tello drone, the video frames would arrive considerably late, so the drone always overshot or undershot. The fix was remembering where it pointed when each frame was taken and turning toward where the person is now. The drone, essentially, predicts where the person will be in the future in order to find its way to the person, and recalibrates constantly to center the person in its camera, making this extension perfect for video-recordings.
  • I had insane difficulty getting into the AMD Cloud. No open ports, SSH blocked, and the platform's tunnel failing with a 403 from a network proxy. I was consistently going over to the AMD Team's office hours, who were graceful enough to help me in generating my own SSH key and also creating my own instance. In the end, starting the tunnel (URL) with the proxy settings removed finally connected the instance to my local base.

What I learned

I learnt the value of consistency. Staying up, late at night, constantly trying to calibrate a drone that lasts a maximum of 2 minutes in the air, was exhausting. However, it was even more gratifying once I saw that the project worked, and that it flew on its own. I was also made aware of AMD Radeon technology. I was not actuated with the AMD Cloud, but when I saw that you could interpolate streams to almost double effective fps, I decided to make it a key factor of my project. It was a good call, as I learned many different instance integration techniques and got many different tips from the AMD team. I also made certain to know the limitations of interpolation technology by keeping real data from real streams separate from the interpolated stream, for greater pools of accuracy. I also really liked to work on the hardware of this project! Utilizing and wiring different pieces of hardware for this project was generally the most fun part for me. I have many different pages of written blueprints made for this project, which all got put to good use.

What's next

A GPS drone for long-range following outdoors, the hand-tilt gauntlet (firmware's already in production, I just need to design the blueprint...), obstacle avoidance on the real drone model, and moving the general brain of the project from the laptop onto the phone. Also producing a custom PCB utilizing KiCad, it's almost done!

Built With

Share this project:

Updates

Submission history