Owing to the International Labor's Day on May 1, 2021 - which fell on the same days of RU Hacks, we wanted to make a product that would eliminate the risk of painters who work at dangerously high altitudes (example : bridges, building roofs etc.,).
We also wanted our product to be first of its kind and crazy - so, we brainstormed to make a Drone which can doodle using a marker and also paint using a spray painter.
What it does
'Air Doodle' is a drone can automatically travel to any spot of the wall that you want and draw a doodle/ paint any figure that the user wants. The user is asked to draw a doodle on the MS Paint like drawing canvas in our web app. The Tello drone uses Computer vision, Machine Learning along with a bit of graph calculations to plot the doodle in the desired place of the wall.
Features in a nutshell :-
Automatic calibration :- The drone automatically calibrates itself by plotting two virtual points on the surface of the wall where the doodle has to be painted. OpenCV is used to calibrate the drone within the points and make its position stable and relative to the position of the two virtual points.
Real-time doodling :- The user can draw any 2 Dimensional figure on the drawing canvas which is provided in the web application. The drone automatically tracks the strokes by the user and replicates the same stroke on the wall.
Doodling from a PNG image :- Using Fourier series, the drone automatically finds the starting and ending points from an image and replicates the same on the wall. This feature is not yet fully implemented.
Saving doodles :- Doodles can optionally be saved to the cloud using DataStax Astra and Google Cloud.
How we built it
We built a web application which is connected to the DJI Tello drone (a programmable drone) over WiFi. The user should connect their PC to the same network of their Drone. Once they are connected, the Python REST API server sends the data from the doodle stokes made by the user in the web application's drawing canvas. Using Computer vision (OpenCV), matplotlib and NumPy, the drone replicates the same strokes over the wall by adjusting its X,Y and Z axis accelerations accordingly.
We made the project using house hold stuff but the project can be commercialized further where each Air doodle drone can be built at around ~ $200. However, please feel free to clone our GitHub repository and use the provided code on your DJI Tello drone to make your own Air doodle!
Challenges we ran into
- Making the drone automatically travel to the desired point on the wall which is selected by the user was a tiring task. We had to do a lot of research on autonomous drones before solving this challenge.
- Changing the X, Y and Z directional accelerations of the drone based on the stroke of the user in the web app was challenging yet an interesting concept to learn.
- The most hardest part was fabricating the drone with a marker and a sprayer fixed with servo motor. We had to reduce most of the components to make sure that the drone could lift them. Since DJI Tello is a cheap and light weight drone, it could not carry both the marker and the sprayer on one flight. So, we had to use only one among them during a flight. We will try to overcome this in future by using a custom built drone.
Accomplishments that we're proud of
- Making our first hack using a drone.
- Trying to reduce the risk of painters who manually paint at high locations like bridges, roof tops etc.,
- Team work and time management.
- Successfully coming up with a project that all our team mates are proud of.
What we learned
- We learnt the most out of RU Hacks. We literally had very less knowledge about drones in the beginning but we're really happy for learning and implementing such an idea within the 48 hour time.
- We learnt how to position the drone by changing it's axis according to the requirement. Luckily, we found a lot of tutorials on how to do it.
- Fourier Series Transformations for finding the paths.
- Using Flask to stream the video feed from the drone and OpenCV on a web port. The streamed video was shown in the web using an iframe.
What's next for Air Doodle
- Adding more color options (currently only one color can be used in drawing).
- Fabricating it better and trying to build a custom drone.
- Using more servo motors to provide more functionalities like controlling the spread of the spray paint etc.,