Fable Flight: Your Personal AI Flight Simulator

Coastal Crosswinds
Cockpit View
Stormy Weather
Desert Run
Cockpit View in Flight
Settings

Fable Flight
Track: Education

Why I built it

Learning to fly a plane is intimidating. I wanted to develop an AI flight simulator so you have freedom to fail, course correct and develop intuitions for this skill at my own pace. You would be surprised how people underappreciate the history and science of air travel. Many people only know the story of the Wright brothers and have limited sense of the skill, discipline and technical sophistication that pilots and aeronautical engineers actually work with.

I suppose binge watching Air Craft Investigation episodes and taking undergrad level physics courses is not enough to teach you how to fly a plane. I still get anxious when turbulence hits, get confused about basic facts even though I understand Bernoulli’s principle/stress-strain relationships and the physics of it all. Current simulators are expensive and don’t adapt to your actual experience level nor push you beyond your comfort zone. I saw this as a perfect opportunity to use OpenAI’s gpt-realtime 2.1 to build something much more responsive so you learn to fly on the fly (pun intended), learn the right protocol for the right situation and have fun along the way. Thinking like a pilot feels like something that should require years of study. But it doesn’t have to! Flying a plane should not feel like a mystery.

What you actually need is the right environment to experiment, fail, try again and fail better. Everyone can become a competent pilot and develop real intuition for how the physics works - without guessing. This is how you build an intuitive feel for airspeed, altitude, cruising, how kinetic energy dissipates, which procedures work in which situations, what a flap does etc. Having real faith in the physics can be just as important as memorising ratios and numbers. AI has the power to dramatically speed up that kind of intuitive, experience-based learning. That’s exactly why companies spend millions of dollars building advanced simulation software. I wanted to bring that same quality of training to everyone by making it far more accessible and affordable. Codex helps me achieve just that.

What I built

Fable Flight is a browser-based AI flight school with a multi-view 3D cockpit, landscape, weather, traffic, missions and a GPT real-time voice assistant to guide you along the way.

Fable = a short story that teaches a lesson or moral.
Flight = the actual flying experience.

The cockpit contains eleven inspectable controls with live values, operating ranges and short contextual lessons. I used Image-to-ThreeJS Typescript based rendering to develop the 3D parts of the aircraft. Learners can practice cold starts, pitot icing and alternator failures. They have to operate the controls themselves.

A Black Box records the important flight events (without storing microphone audio). After the flight, the debrief points directly back to the recorded moments instead of giving generic advice. This was the hardest part of the build.

An early version would sometimes answer before the latest aircraft state had reached the real-time session. The reply sounded plausible, but it was based on an older snapshot. I fixed the pipeline so every turn receives a fresh, bounded view of the aircraft state, warning state, current lesson and selected control. GPT-5.6 explains why it matters and suggests a useful next exercise.

How I used Codex

Codex helped me build and inspect the telemetry contracts, cockpit interactions, real-time wiring, browser behaviour, and verification tests. It was especially useful for tracing the stale-state bug across the browser and data channel instead of treating it as a prompt-engineering problem. Wind physics, Flight Path and the ability to actually develop some mastery of flight instead of gaming or "reward hacking" (human analogue) the flight controls was a welcome challenge. I needed to change the /goal repeatedly to achieve this. No matter how long the /goal ran tweaks were definitely needed to achieve such a system. Since I submitted many projects at once, I used a singular Thread to manage the /goals for all other threads including the thread to make Fable Flight: 019dc003-290b-7652-84ae-a2c44110087f or the deeplink here: codex://threads/019f4665-cd0c-70c2-98de-6ae26ee68969

What judges can try

Fly one circuit and deliberately make it unstable. Open the debrief, ask the instructor what went wrong and then fly a second circuit using the new lesson. Microphone helps. Use the camera to go to a cockpit view. Follow the map directions and steer the plan accordingly. You will receive a care package which you must drop at the city. Biggest learning (which the AI instructor will tell you) is to lower the nose to gain air speed, dont be too greedy to gain altitude without sufficient air speed otherwise you will descend and crash. But don't give up if you crash! Its normal. After a few tries you will feel a rewarding sensation!

Fable Flight may require server-side OpenAI setup for live GPT Realtime voice but I already set it up in env variables in Vercel. It only calls /api/realtime/health and /api/realtime/client-secret. I got my friends to try it and they love it!

Current status

The browser simulator, cockpit lessons, Black Box and grounded instruction pipeline are all working. Fable Flight is educational software built mainly during the OpenAI build week. It is not certified flight training. Majority of the work is done post July 13 with thread controls, multi day goal and sub goals with sub agents and unique workflows from X/custom made. I plan to expand this further by connecting an adaptive instructor/analytics layer that will scale flight training into classroom workflows, route mission assignments by learner skill and continuously refine coaching from real telemetry so each pilot improves with measurable progression.

Github Main: https://github.com/Arnie016/flight-simulator-fable5