Halo | Devpost

Final prototype of our smart helmet, Halo.
Building and configuring our Raspberry Pi 5 and accessories needed
visual representation of object detection

Inspiration

In 2022, 1,105 cyclists were killed on US roads, the most since 1975. Florida, where we built this, is consistently one of the deadliest states in the country to ride a bike. Many of these crashes start behind the rider: a driver who doesn't see them, or who passes far too close. Cars have had blind-spot monitoring for years, and a Waymo sees 360° with fused sensors. The cyclist next to it, the most exposed person on the road, has a mirror the size of a coin. We built Halo to give riders the awareness a modern car has, inside the one thing every rider already wears.

What it does

Halo is a smart bike helmet that watches the road behind you and warns you about what's coming before it reaches you, so you don't have to turn your head.

A rear camera and side ultrasonic sensors track vehicles approaching from behind and beside you. When a car enters your blind spot, cuts into your lane, or closes in fast, Halo warns you on the side it's coming from:

an arrow on a transparent OLED in your peripheral vision (outline = caution, flashing = danger)
a beep panned to that ear in your earbuds
a spoken callout like "Car left!"

Close passes are measured, not guessed. The side sensor reports the real gap ("passed at 0.32 m"), which is the number that matters under Florida's 3-foot passing law.

Every serious alert becomes an incident report. The helmet saves a clip from 6 seconds before to 4 seconds after, attaches the measured gap and warning time, and Gemini writes a short summary of what happened. The Halo iOS app tracks your ride, shows the helmet's live safety state and a 3D "rear-view mirror", and keeps your safety history. A front camera adds a forward BRAKE warning demo for obstacles ahead.

Technical Deep Dive

Our core rule: nothing that keeps the rider safe waits on the internet. Detection, alerts and fail-safes all run on the helmet. AI only runs off the critical path.

Perceive: YOLO26n, exported to NCNN at 320 px, runs on a Raspberry Pi 5. The rear camera is mounted upside down, and the camera rotates the image itself at no cost.
Track and measure: each vehicle is tracked. Time-to-contact comes from how fast its bounding box grows, so no depth sensor is needed, and it stays correct when the box is cut off at the frame edge. Lateral position and velocity predict each car's path, so a car cutting into your lane is flagged before it arrives.
Fuse: side ultrasonic sensors are median-filtered, and a wall or parked car fades into the background. When a tracked car leaves the camera's view beside you, the side sensor takes over and measures real clearance. A danger alert requires the camera to confirm it's a vehicle. An echo alone can only raise a caution, because it can't tell a car from a pole.
Decide: a tiered alert policy (tracked → approaching → blind spot → danger) with hysteresis keeps alerts from flickering. One shared state drives the OLED, earbuds and phone, so every output always agrees.
Fail safely: the Pi sends the Arduino a heartbeat every 250 ms. If it goes silent, the rider feels a warning buzz and the helmet falls back to a plain bike light. If the camera drops, the OLED shows an X, Halo says "rear camera offline", and the side sensors keep warning. Gemini calls are async, budgeted, rate-limited and circuit-broken, so if Gemini is slow or down, the helmet keeps working.
Record and explain: frames sit in an in-memory ring buffer. A danger alert writes an H.264 clip in a low-priority background thread. Gemini receives 6 key frames plus the measured facts and returns structured JSON (classification, severity, vehicle, maneuver, summary). It's instructed to treat the sensor numbers as ground truth, and the API key never leaves the helmet.
Show: the Pi streams ~700-byte JSON snapshots over server-sent events, and a bundled three.js scene renders the 3D mirror on any phone, even on a hotspot with no internet.

How we built it

We built Halo from scratch: hardware, firmware, the vision pipeline and the app.

Hardware

Raspberry Pi 5 (main compute)
2× Raspberry Pi cameras (rear, mounted inverted, and front)
Arduino Uno (sensors, haptics, fail-safe watchdog)
HC-SR04 ultrasonic sensors, angled back on each side
1.51" transparent OLED (SSD1309) as a heads-up display
Vibration motors and Bluetooth earbuds for audio alerts

Software

Python perception pipeline: YOLO + NCNN, OpenCV, Picamera2/libcamera
Camera + ultrasonic sensor fusion with camera-to-sensor hand-off
Serial protocol between Pi and Arduino with a heartbeat watchdog
Gemini for incident analysis and "what's behind me?" voice questions
three.js 2.5D rear-view mirror, streamed live from the helmet
SwiftUI iOS app with MapKit navigation, ride tracking and safety history
A traffic simulator that runs scripted scenes (close passes, cut-ins, tailgaters) through the real pipeline, plus test suites for every module

Challenges we ran into

Our biggest challenge was speed on a tiny computer. A warning that arrives a second late is useless, and everything, including detection, tracking, sensor fusion, the 3D view and incident recording, has to run on one Raspberry Pi strapped to a helmet. Our first attempts couldn't keep up.

We had to make every millisecond count. We shrank the detection model and converted it to a format built for the Pi's processor. We rotate the upside-down camera image in the camera hardware instead of in software. Video encoding runs at low priority in the background, and the phone gets tiny data snapshots instead of a video stream. Gemini was the biggest trap: an AI call can take seconds. So we moved it off the safety path entirely. It never delays an alert, and if it's slow or offline, the helmet keeps working without it.

Another unique challenge we faced happened before we wrote a line of code: we threw out our idea. We came to ShellHacks planning a fintech project, a pair of smart glasses that read price tags as you shop and showed your budget in your field of view, like "Honey for the physical world." Once we saw the challenge tracks, we realized a camera, a transparent display and on-device AI could do something far more important than save money: keep someone alive.

Accomplishments that we're proud of

A working helmet that detects and warns about cars behind you, fully on-device and offline
Measured close passes, not estimates
One safety state that keeps the OLED, earbuds and phone in agreement
Incident reports where the AI explains the clip but never overrides the measured numbers
Layered fail-safes: watchdog, camera fault handling and graceful AI degradation
A simulator good enough to develop and demo without a single car

What we learned

We learned how different real sensors are from the ideal version: noise, latency, mounting angles and cables all change behavior.

We also learned:

Which numbers to trust, and how to design around the ones you can't
Why safety-critical logic has to be separated from AI and the network
How to fuse sensors that are each wrong in different ways
How much hardware integration there is, from camera numbering to audio drivers

What's next for Halo

Road testing, and a proper integrated shell with battery and weatherproofing
A wider-angle rear camera and radar for better range at speed
GPS-tagged incident reports riders can share, and an anonymized close-pass map to help cities find dangerous roads
Full incident video playback and reporting in the iOS app

The goal is to take Halo from a hackathon prototype to something every rider wears without thinking about it.