SkySentry

Inspiration

In 2009, geese knocked out both engines of US Airways Flight 1549 minutes after takeoff from LaGuardia, forcing the "Miracle on the Hudson" landing. In 2018, drone sightings shut down London's Gatwick Airport over three days and disrupted about 1,000 flights. Big airports fight these threats with bird radar and counter-drone systems that cost a fortune. Small airports near us, like Princeton Airport and Trenton-Mercer, can't afford them and rely on people looking out the window. For a hackathon themed "Into the Skies," we wanted to give those airports an early warning built from something they already have: a phone.

What it does

SkySentry turns an ordinary phone into an early-warning system for birds and drones near a runway.

Sees birds: the camera counts birds and tracks which way they're flying, including whether they're heading toward the runway.
Hears drones: the microphone recognizes a drone's buzz, even when it's too small to see.
Rates the risk: green, yellow, or red, shown instantly on a physical Arduino warning light.
Warns the tower out loud: red alerts are spoken by an AI voice ("Caution, flock of eight birds crossing the runway, heading east").
Writes the report: Gemini checks the snapshot and writes an incident report in seconds.
Keeps a history: every detection is stored, so the tower dashboard shows hourly charts, a map of alerts, and a chat where you can ask "When were birds worst today?"
Protects evidence: drone reports get a tamper-proof stamp on the Solana blockchain.

How we built it

We split the project into four parts, each with one owner and its own folder in the repo, and agreed on a shared event format first.

Detection + hardware (Vivan): TensorFlow.js runs COCO-SSD in the browser to find birds, and our own frame-to-frame tracking adds count and direction. A Google Teachable Machine audio model trained on background noise, drones, and birds listens for drones. A risk engine sets green, yellow, or red and drives LEDs on an Arduino and breadboard as a physical warning light.
Database + data (Pranav): Node.js, Express, and Socket.IO hosted on DigitalOcean App Platform, with every API key in environment variables. Tiger Data (PostgreSQL for time-series) stores each detection in a hypertable, with a continuous aggregate powering the hourly charts. MongoDB Atlas stores full incident reports with snapshots.
AI + voice (Shreyansh): Gemini, via Google AI Studio, verifies yellow and red snapshots, writes incident reports, and answers chat questions using only our stored stats. ElevenLabs turns red alerts into spoken announcements. Drone reports are hashed with SHA-256 and stamped on Solana devnet using the Memo program.
Dashboard + presentation (Atharv): a live tower dashboard with a risk meter, alert feed, Leaflet map of Princeton Airport with alert pins, Chart.js hourly charts, and the "Ask SkySentry" chat.
We used GitHub Copilot throughout to code faster.

Challenges we ran into

A loud room vs. a drone model: hackathon noise confused the sound model, so we recorded background noise in the room itself and required 80% confidence held for a full second before alerting.
Small birds: the off-the-shelf model struggles when birds are tiny in the frame, so we tuned thresholds on test clips.
Real-time end to end: getting camera → server → databases → AI → voice → dashboard working in seconds meant fixing [the bug you actually hit].
Hardware meets software: getting the Arduino to react to the browser's risk level took [what you actually did].
Four people, one repo: with AI-written code coming from four laptops, separate folders and an agreed event format kept us from overwriting each other.

Accomplishments that we're proud of

A full pipeline from a bird in the camera to a spoken warning in the tower in about [X] seconds.
It caught [X of Y] test clips, with [Z] false alarms on clips with no birds.
A physical warning light, so the alert isn't stuck on a screen.
Six sponsor technologies, each doing a real job: ElevenLabs, Gemini, Tiger Data, DigitalOcean, MongoDB Atlas, and Solana.
All built in 24 hours with a phone, an Arduino, and free-tier tools.

What we learned

How to run AI models directly in the browser and add tracking on top of a detector.
That sound catches what a camera can't, and that training data from the real environment matters.
When a time-series database fits better than a document database, and vice versa.
How to use a second AI model as a check instead of trusting one model alone.
How to connect hardware to a web app.
That agreeing on interfaces first is what makes four people working in parallel possible.