Air traffic controller -> pilot simulator

Teacher assistant and simulator for any kind of flight scenario you can think of.

Comment

Inspiration

Airplanes are fun! Looking for niche (unexpected) industries that might want generated audio. ChatGPT suggested training simulations :)

A lot of material online from interesting flight scenarios to listen in on, but for the regular person, simulations and trainings could be expensive.

What it does

It let's you participate in a flight scenario and practice your air traffic controller --> pilot communications skills where you are the student pilot.

How we built it

  • Got some ATC -> pilot communication audio clips + audio transformations,
  • Ran speaker classification and voice / talk activity detection on the files.
  • Cut all audio for each speaker and merged the files per speaker for Elevenlabs voice cloning

  • Cloned ATC voices.

  • Built a simulator interface with audio input and output + scenario building using OpenAI.

Challenges we ran into

  • All the smaller things in UI frontend development :P
  • Speaker classification using open source models is not perfect (yet)
  • Running multi-step pipelines is time consuming to run manually.

Accomplishments that we're proud of

Getting something end to end working involving audio, scenario building, creating a audio dataset from scratch.

What we learned

There are interesting possibilities to explore in audio narration etc.

What's next for Air traffic controller -> pilot simulator

  • Different level of difficulties.
  • Teacher / advisor mode, to learn.
  • Fun-mode, say whatever and it'll still act as a ATC controller.

Built With

Share this project:

Updates