Haptix

Inspiration

As young college students looking for apartments next year, we took many virtual tours of nearby places to see if they fit our expectations. However, this made us wonder how blind people were able to tour houses virtually. We wanted to make a product that blind people can use for virtual house tours, giving them more independence in how they live their life.

What it does

Blind Virtual Tour helps blind and low-vision people explore a home’s layout through touch and sound. Users move a virtual avatar using their iPhone as a controller. Distinct vibrations communicate walls, doorways, stairs, and floor textures, while spoken descriptions and spatial audio provide context and direction. Users can follow a guided path, step off it to explore freely, and rejoin when they want guidance. A connected computer displays the floorplan, live position, and narration so others can follow along.

How we built it

Using AI-enforced coding methodologies, we built the iPhone app with Swift and SwiftUI, using Core Haptics for vibration patterns and AVFoundation for speech and spatial audio. VoiceOver integration lets users hear announcements through their familiar accessibility settings. Python scripts convert floorplan images into structured JSON data, with room information and tour routes annotated manually. The app uses that data for movement, collision detection, and navigation. An iPhone-hosted web server sends live updates to an HTML, CSS, and JavaScript desktop viewer.

Challenges we ran into

The biggest challenge was making movement understandable without visual feedback. Open-ended navigation can leave users disoriented, so we added a fixed path with the option to leave and rejoin it. We also needed haptic patterns that felt distinct without becoming overwhelming. Balancing vibrations, spatial audio, and narration required careful choices about what information each channel communicates. Floorplans also contain incomplete information, so we had to distinguish known details from missing data.

Accomplishments that we're proud of

We created an interactive experience that makes a visual floorplan explorable through multiple senses. The app supports two house layouts, movement between floors, wall collisions, guided narration, and customizable feedback. We’re especially proud of combining structured guidance with independent exploration. We also redesigned the desktop viewer while preserving its existing movement and connection logic.

What we learned

Accessibility shapes the entire interaction, from movement controls to how much information is spoken at once. More feedback does not always mean a better experience; concise descriptions and recognizable cues can make exploration easier to follow. We learned that predictable navigation and a clear way to recover are essential. We also learned to represent missing floorplan information honestly instead of filling gaps with assumptions.

What's next for Haptix

Our next priority is testing with blind and low-vision users to evaluate navigation, haptic recognition, and how well the experience helps them understand a home. We want to add destination-based guidance, more control over narration, and easier importing of additional floorplans. Longer term, we hope to support verified furniture and fixture layouts, accessible home comparisons, and integration with housing listings so more people can explore potential homes independently.