Eye Spy

Inspiration

A walk to the polyclinic should be straightforward. But for a senior, a wheelchair user, or someone with low vision, a flight of stairs, a missing kerb ramp, or broken paving can turn a short journey into a difficult one. These barriers are visible on the ground, yet often missing from the information used to plan routes and improve neighbourhoods. Knowing that a footpath exists does not tell us whether someone can use it. That gap inspired Eye Spy: a way to help people navigate barriers today while giving planners the evidence to remove them tomorrow.

What it does

Eye Spy will connect personal navigation with community accessibility mapping. Our smart glasses system will observe the path ahead, identify potential obstacles, and provide spoken warnings and directions. Paired with a phone, it will help the wearer find a step-free route to destinations such as polyclinics and Active Ageing Centres. Each walk will also contribute information about the neighbourhood. Detected barriers will be mapped to footpaths and brought together in a planning dashboard, helping agencies prioritise improvements according to the severity of each barrier and the people and journeys it affects.

How we built it

We are designing Eye Spy around two connected workflows: assistance for the wearer and actionable information for planners. For the wearer, the glasses will stream video to a vision model on our server. The model will flag accessibility barriers, and the glasses will deliver spoken guidance. For planners, our Databricks architecture will turn observations into structured evidence:

• Unity Catalog Volumes and Delta tables will store blurred video and barrier detections.

• Lakeflow will associate observations with footpaths and merge repeated sightings.

• A scoring pipeline will rank gaps using severity, nearby senior populations, destinations, and footfall.

• MLflow will track model performance.

• An AI/BI dashboard will present the prioritised improvements.

We will combine these observations with public datasets covering footpaths, crossings, population, care destinations, and transport.

Challenges we ran into

Our main challenges will centre on reliability and usefulness. The system will need to recognise barriers under changing lighting, weather, and pedestrian conditions. It will also need to locate observations accurately and distinguish repeated sightings from separate problems. Spoken guidance must be timely and clear without overwhelming the wearer. Meanwhile, the planning dashboard must turn many individual detections into priorities that agencies can understand and act on.

What we expect to learn

We expect to learn how far the shortest route can diverge from an accessible one, since accessibility depends on details that a basic street map leaves out. We also aim to understand what it takes to move beyond detection. Identifying an obstacle only matters if it leads to an understandable warning, a suitable alternative route, or a practical infrastructure decision.

Project members:

Isaac Tan Yu Yang, Year 1 studying Business Artificial Intelligence Systems at National University of Singapore, [email protected] Joshua Tan Yu Jie, Year 4 studying Computer Science at National University of Singapore, [email protected]