C.H.U.D (Collaborative Hud for Unified Data)

GUI
Drone_PCB
Software Stack Diagram

Inspiration

We built a physical hardware drone and wanted to explore how software could be used to control it. This project gave us the opportunity to bridge our hardware with voice-driven software control.

What it does

Our system converts voice messages into drone commands. The voice input is transcribed into text using AI, then parsed with Python and Regex to generate commands that the drone can execute.

How we built it

The hardware was developed over four months using Altium, with an ESP32 as the main controller. For the software side, we leveraged AI tools to assist in building a voice-to-command pipeline, integrating transcription, parsing, and execution into a functional workflow.

Challenges we ran into

One of the biggest challenges was integrating map information into the UI. We aimed for a dynamic and interactive experience, which required continuous communication with Gazebo. The lack of a standardized coordinate system led to occasional overshoot and made synchronization more difficult.

Accomplishments that we're proud of

We created a clean, user-friendly map UI using Streamlit, making it easy to visualize drone commands and positions in real-time.

What we learned

We learned how to use OpenAI Whisper to convert audio messages into usable text, and how to leverage a Python Regex parser to interpret those commands for the drone.