Here is your updated project description with standard Markdown formatting applied to the equations and the clarification regarding the smart glasses added to the text.

Inspiration

Navigation still makes you look away from the road: glance at a map, figure out which of five lanes "keep right" means, look back. AR head-up displays exist, but only in luxury cars. We wanted a dashboard tablet or smartphone that sees the road like you do and paints directions directly onto it—serving as a physical substitute and stepping stone while waiting for consumer-ready AR smart glasses like Meta's upcoming hardware. We set one hard rule from day one: Ahead Up only informs, it never drives. It never steers, brakes, or accelerates, no LLM makes driving decisions, and every alert is measurable ("Vehicle ahead: 8.4 m").

What it does

Ahead Up turns a Galaxy Tab S9 (or Android smartphone) on the dashboard into an AR driving copilot:

Lane arrows painted on the road. Your target lane is green. If you're in the wrong lane for an upcoming exit, your arrow turns red and the target lane blinks.
Turn-by-turn navigation with place search ("Where to?"), from a real routing engine using live GPS.
Following-distance alerts on the car in your lane only, with a measured distance.
Traffic lights, pedestrians, and speed-limit signs, plus voice cues like "Move to the right lane for the exit, check for cars."
Honest degradation. If perception data is stale (older than 500 ms), object alerts are switched off and a banner says so.

How we built it

Perception (Python, RTX laptop): YOLO26 fine-tuned on BDD100K for detection, BoT-SORT for tracking, Depth Anything 3 fused with flat-ground geometry for distance, TwinLiteNet+ for lanes, and an Autoware classifier with HMM smoothing for traffic lights. It runs as a two-lane realtime engine: a fast lane handles every frame, a slow lane handles depth and lanes.
Navigation (Node.js): Our routing engine turns the destination, GPS, and Google Routes data into maneuvers, distances, and ETA.
Mobile Hardware (Kotlin, Compose, CameraX): Streams JPEG frames from a dashboard-mounted tablet or phone to the laptop and projects arrows onto the road using the same math as the server, redrawn at the display rate. This mobile setup acts as our stand-in platform while next-gen spatial display hardware matures.
Glue: One WebSocket protocol with JSON Schemas and golden samples that both the Python and Kotlin sides test against, plus a JVM "fake tablet" so we could test end-to-end before the hardware was ready.

On flat ground, an object's distance comes from where its box touches the road:

Z = (f * H) / Δy

where f is the focal length in pixels, H is the camera height, and Δy is the pixel gap between the box bottom and the horizon. Time-to-collision comes from how fast a box grows:

TTC ≈ Δt / (s - 1)

where s is how much the box height grew over the time Δt between frames.

Challenges we ran into

Syncing many realtime processes at once was the hardest part. The camera stream, two GPU model lanes, the navigation engine, text-to-speech, and a 60 fps AR renderer each run on their own clock. An arrow drawn from a 200 ms-old frame lands in the wrong spot. To fix that:

Credit-based flow control: At most 2 frames in flight, frames without a credit are dropped instead of queued, and every frame gets exactly one answer. This got capture-to-result latency down to about 50 ms.
SIM look-ahead: Results arrive about 150 ms before the video frame they belong to is shown.
False alerts: Parked cars kept being picked as the "lead vehicle". We added a corridor of ±1.3 m around the car's own path, which cut false TOO CLOSE events roughly in half.

Accomplishments that we're proud of

A full working pipeline: camera, GPU perception, navigation, then AR and voice on a mobile device acting as our glass-substitute HUD.
Every model chosen by measurement on real driving data. For example, traffic lights are 87% accurate on lit lights, and red is read as green only 0.6% of the time.
A shared protocol contract with 130+ tests, so three people could build in parallel and still integrate cleanly.
Honest by design: "UNKNOWN" stays unknown, and a green light is never shown as permission to go.

What we learned

None of us had touched AR, WebSockets, Kotlin/Android, or realtime GPU pipelines before this.
In realtime systems, dropping frames and predicting forward beats queuing them.
Write the contract first: The protocol spec saved us hours during integration.
Measure, don't assume: Camera focal length mattered more for distance accuracy than which depth model we used.

What's next for Ahead Up

Port to true AR Smart Glasses: Deploy on the Meta Horizon OS / Meta VR SDK, transitioning our spatial UI from the dashboard tablet/phone directly to the user's field of view on Meta's upcoming smart glasses.
Integrate driving lessons: Coaching on following distance, lane discipline, and traffic-light awareness, with a summary after each drive.
Real windshield drives with real GPS and speed-aware following distance.