Aerovox

Selected Plane and Airport
General aerial view
Hardware component
Turbulence Aversion
Multi-Airport Landings

Inspiration

Being overwhelmed is not a great feeling to have. This is an issue that greatly affects pilots, who have to constantly intake a variety of information on other planes in the area, the current weather conditions, plane resources and fuel, etc. We wanted to help ease the workload of these pilots.

What it does

AeroVox is a real time AI copilot made to assist with cockpit operations by using ADS-B data to create an interactive, hands free environment. The system uses a physics-based extrapolation pipeline in order to accurately render planes in real time onto an airspace overlay.

By bridging natural language processing directly into the systems, AeroVox allows pilots to manage navigation, reroute effectively, and quickly make last minute inflight diversions entirely through voice commands, greatly increasing efficiency.

How we built it

We used a react stack for the system interface, with a Leaflet map overlay. We took airport and runway data from AirportDB, then plotted them onto the map. Live airplane data was obtained using a ADSB.lol API, including location, elevation, and speed. Wind and turbulence data was taken from Open Meteo to allow detection for variance and rerouting.

The natural language processing and voice output was done using Elevenlabs. We used the speech to text feature in order to process possible commands given by the pilots, then used text to speech to relay any messages and responses to the pilot. Additionally, there is translation support for the text to speech output language.

Challenges we ran into

Choosing good APIs that fit the purpose of our project was a bit difficult. Due to the volume of data we would need, many of the APIs would have to have a generous free plan. After enough digging, we were able to find APIs that fit our needs, but still had to scale down the scope so we would not run out of calls.

When creating the physical model of the plane, we planned on using servo motors to simulate the yaw, making it mirror the heading in the software simulation. However, we ran into a hurdle when we realised that the servos could only turn in a 180° range of motion. As a workaround, we decided to use a stepper motor, which has a much more flexible 360° range of motion, allowing for us to effectively simulate the heading of the plane.

Accomplishments that we're proud of

Integrating both hardware and software something that most of us had little to no experience in. Figuring out connecting the React files to Arduino files using a Python bridge was something that took some time to get working, but was very rewarding to see in action once it was completed.

In terms of working efficiency, we were able to set up the base of the project (a map with live data of planes, airports, and runways) in quite a short amount of time. This enabled us to focus more effort into the hardware and the core features of our project.

What we learned

We learned that the ideation phase is very important to a project. When we were spitballing ideas, we sunk a lot of time exploring rabbit holes that didn't actually matter much in the grand scheme of the idea. This resulted in us starting the project a lot later than it could have. We also didn't come up with a clear MVP before diving into the project, spending more effort deciding on what to make when. In the future, better pre-planning can be implemented to make the development process more streamlined.

What's next for Aerovox

Aerovox can be expanded from just commercial usage to also serving personal and military uses. Private aircraft pilots can use it in the same way airlines do, to take some of the mental effort of doing the basic tasks out. For militaries, drone operators can use voice control to control more than one drone at a time, increasing the possible quantity and quality of group drone expeditions.