Front View
Close-up
Software Demo

Beacon

Inspiration

In the U.S., more than 7 million people have vision loss that glasses or contacts can't correct, including about 706,000 people who are blind. Accessibility barriers are a routine part of their day. In an American Foundation for the Blind study, 21% of participants ran into web accessibility barriers every day, and 28% hit inaccessible mobile apps daily. A follow-up diary study found that blind and low-vision participants averaged 12 barriers per week and spent 2.4 hours dealing with them. That's a lot of lost time! Beacon focuses on the barriers in the physical world.

What it does

Beacon gives users a simple way to access visual information in the physical world. Press one button, take a photo, and receive an audio description of what's around you — hazards, objects, signs, text, spatial relationships, and answers to specific questions you ask about the scene.

Instead of just describing an image, Beacon tells you what you need to know to navigate independently. It:

Identifies hazards and obstacles in your immediate path
Reads signs and text aloud
Describes spatial layout (what's to your left, how far away something is)
Answers follow-up questions about the scene
Responds in a natural voice, not a robotic one

How we built it

We split the project into two phases: AI integration and software development, then hardware and wearable integration.

We started with a minimal demo: a website hosted on Vercel that integrates the Grok API on the backend. Using a laptop or phone camera, we showed that the site could capture a scene and prompt Grok to describe it audibly and succinctly.

From there we focused on comfort and speed. We added audio cues for low-vision users, and improved response time with a layered approach — sending a small image first, then a larger one only if more detail was needed. We also swapped the robotic default voice for ElevenLabs text-to-speech to make the output sound natural.

For hardware, we used a Raspberry Pi 4B with a 1080p megapixel camera to capture photos and send them over Wi-Fi to our backend for processing by Grok. The backend returns an MP3, which we play back through either speakers (for demos) or UAJ earbuds.

Challenges we ran into

This was our first time using an API to retrieve information, especially visual data. We're both primarily hardware engineers :)

Getting into the Raspberry Pi was tough. SSH wouldn't work on JHU's Wi-Fi, and the Raspberry Pi OS Imager wasn't writing to the SD card or partitioning it correctly. We fixed it by switching to dorm Ethernet, then figured out how to use a hotspot.

Connecting the two sides was the other big one. To get the photo from the Pi to Grok and a valid MP3 back, we needed somewhere to store the image. We used Vercel Blob, which meant learning Vercel and website deployment from scratch!

Accomplishments that we're proud of

We took on a VERY (all caps VERY) ambitious software and hardware project that we had almost no idea how to build. We just thought it would be cool!
With little software experience between us, we learned Node.js, Vercel, and React.
We learned a lot on the hardware side too. We configured Wi-Fi by typing the path into the OS files by hand, and we found a trick that worked when nothing else did: an empty ssh file makes the Pi enable SSH.
We got speakers, earbuds, a mic, and a camera all working together.
Pressing one button and watching the whole process run, from photo to spoken description, was an unbeatable feeling!