SkyTrace

Dashboard UI
OpenCV Frame Capturing Yellow Plane
Flight Log Video Saving
Trajectory Graph
Statistics
Flight Logs

Inspiration

We all love paper airplanes. They make flight something anyone can experience with a sheet of paper. But after a throw, it’s hard to explain why one flight felt smoother than another or whether a design change actually helped. Inspired by the hackathon’s "Into the Skies" theme, we created SkyTrace to turn paper airplane throws into measurable experiments. We wanted to combine computer vision, physical hardware, and an interactive website to make testing airplane designs engaging and accessible.

What it does

SkyTrace tracks a yellow-marked paper airplane through a webcam and displays its flight path on a live dashboard. It scores the smoothness of the camera-visible flight, measures visible airtime, and estimates the short-term trajectory from observed motion. An Arduino LCD displays the final score, while an optional ElevenLabs voice coach delivers spoken feedback. Users can create plane profiles, organize flights by dragging them into categories, rename throws, and compare previous trajectories as transparent overlays. Each newly logged flight can also include a short video with tracking annotations for reviewing the actual throw. Recordings stay local and session-only, with a storage limit. Throws shorter than half a second are excluded from history to reduce accidental entries.

How we built it

We built the tracking system in Python using OpenCV and NumPy. Color detection identifies the airplane’s yellow markings, and position smoothing reduces detection jitter. We use the recorded positions and timestamps to draw trajectories, calculate a smoothness score, and produce a simplified motion forecast. Python sends the final score to an Arduino over USB serial, and the Arduino displays it on a 1602 LCD. Our local Python server connects the tracker to a dashboard built with HTML, CSS, JavaScript, and SVG. We integrated the ElevenLabs text-to-speech API to turn score-based feedback into spoken commentary. For video review, we capture annotated frames and encode short H.264 clips in the background. Saved clips are kept in memory, with older recordings removed when storage reaches its limit.

Challenges we ran into

Reliable tracking was one of our biggest challenges. Lighting, motion blur, and similar background colors can affect detection, so we experimented with colors and adjusted our filtering and smoothing. We also had to distinguish between a plane leaving the camera view and actually landing. Our dashboard freezes completed flights instead of continuing to update their graphs, and it labels forecasts separately from observed measurements. Keeping the website, Arduino, and voice feedback consistent required careful handling of flight results. We fixed cases where commentary could reference an outdated score and made sure replaying a previous flight would not change the current result. Finally, fitting profiles, history, comparisons, and replay into one sidebar took several iterations. We redesigned it into separate Flights and Planes tabs with contextual controls.

Accomplishments that we're proud of

We’re proud of connecting a physical paper airplane throw to a complete feedback experience: live tracking, a trajectory graph, a smoothness score, an LCD result, and spoken commentary. We also built tools that make repeated testing useful—not just entertaining. Plane profiles, comparison overlays, and recorded clips let users review different designs and see how their flights differ. Most importantly, we kept the project honest about its limitations: camera-relative measurements are not real-world distances, and a predicted path is not a measured landing.

What we learned

We learned that computer vision involves much more than detecting an object. Stable tracking requires careful decisions about noise, timing, smoothing, and when a flight begins and ends. We gained experience connecting Python software to Arduino hardware, building a live dashboard, integrating an API securely, and encoding video without blocking the tracking loop. We also learned how much interface design affects usability. Showing fewer controls at once made SkyTrace easier to understand without removing its capabilities.

What's next for SkyTrace

Our immediate next step is testing real throws under different lighting conditions, camera positions, and airplane designs, then fixing any remaining bugs. Beyond the hackathon, we’d like to add tracking-confidence indicators, camera calibration for real-world measurements, and more robust airplane detection that does not depend on colored markings. We also want to improve the prediction model and expand design comparisons, helping users move from “that throw looked better” to understanding how their airplane’s design and launch affect its flight.