ScanAssist

Inspiration

Technology moves fast, and seniors shouldn't be left behind. The idea started with a simple, familiar scene. Someone's parent or grandparent, standing in front of a blinking appliance, unsure which button to press, and no one nearby to ask. Manuals are too small and too technical. Calling a family member feels like a burden. Searching online assumes a level of comfort with technology that many seniors simply don't have.

We kept coming back to one question: what if the answer could come from something seniors already carry, a phone camera, paired with something that actually explains itself in a way they can follow?

What it does

ScanAssist is an AI powered app that turns a single photo into clear, step by step guidance for using everyday devices. Point your phone at a refrigerator panel, a remote, or a microwave, and ScanAssist tells you exactly what to do next, in plain language and out loud.

• Open the app and take a photo of the device or button you're stuck on
• AI identifies what it's looking at and confirms it with you in plain language
• You choose what you need: how to use it, how to fix a problem, or just what it is
• The app walks you through one step at a time, with large text and a voice that reads each step out loud
• You can ask a follow up question at any point, by typing or speaking

Every design decision traces back to the same standard: if it isn't obvious in three seconds, it's too complicated. No nested menus, no small text, no jargon.

How we built it

We didn't want to build this from assumptions, so we started with people, not code.

• We visited a senior living community in Baltimore and spoke directly with residents and staff about how they actually handle confusing devices today
• We printed and distributed a short survey to gather input from residents who preferred writing over conversation
• We interviewed family members and caregivers, including our own parents, to understand the experience from both sides: the senior trying to use the device, and the family member who usually gets the phone call for help

On the technical side, we used CodeX to accelerate development and prototyped our core flows in Figma Make to quickly test different layouts before settling on a senior friendly design. The app integrates GPT's API for image recognition and natural language step generation, paired with a text to speech layer so every instruction can be read aloud.

Challenges we ran into

The hardest part wasn't the technology. It was figuring out what seniors actually wanted, as opposed to what we assumed they wanted.

• Some features we thought were helpful, like confirming every single step with a photo, felt like extra hassle to the people we interviewed
• Voice guidance mattered far more than we initially expected, while some visual details we designed carefully went unnoticed
• Balancing simplicity with usefulness was a constant negotiation. Every screen we removed made the app easier, but we had to make sure we weren't removing something someone genuinely needed

Accomplishments that we're proud of

• We left the building and tested our idea on real people instead of guessing. Talking directly with residents at a senior living community shaped almost every decision we made after that
• We designed an experience genuinely built around accessibility first, with voice guidance, large text, and a flow that needs no prior tech experience to use
• We took feedback seriously enough to cut features we liked, once we learned they added friction instead of removing it

What we learned

Listening to real residents reshaped our product more than any design discussion we had as a team. We learned that the features we assumed would matter most weren't always the ones users cared about, and that simplicity is a discipline, not a default. We also learned how differently a senior and a caregiver can experience the same problem, which shaped how we think about who this product is really for.

What's next for ScanAssist

This is just a starting point. We want to keep testing with real users to understand which features genuinely reduce friction and which ones just add noise. Looking ahead, we're especially interested in building family and caregiver features, allowing a family member to link their account to a senior's profile, check in on what they've been using, and step in when extra support is needed. Technology

Built With

• browser-camera/file-apis
• canvas-api
• css
• github
• html
• javascript
• localstorage
• node.js
• openai-responses-api
• web-speech-api