We're making an app specifically targeting a group of people who usually get left behind when it comes to technology. Nowadays, the hottest and trendiest technologies are all about wild moonshots and big visions: what looks the coolest and what lets us see the world in different ways. However, it's really easy to forget that a part of the population often gets left behind in all the hype: the visually impaired.
We wanted to create an accessibility app that makes it easier for partially sighted and blind users to have a great experience on Android mobile. By combining the technology of TPad Tablet and Android mobile, we were able to make the visual mobile experience more accessible by delivering specific forms of haptic feedback when users ran their fingers across the screen. TPad's technology allowed us to create 3D textures on the 2D screen via vibration physics, which could be altered and filtered by factors such as color, exposure, and saturation.
For our project, we were able to deliver various tactile sensations, filtered by hue and saturation, to help visually impaired users navigate Android mobile in two key ways: 1) distinguishing between app icons on the screen, and 2) distinguishing between different button placements on the screen.
Our app, named AccessiApp, bundles together a variety of demos to demonstrate just how useful tactile feedback has the potential to be in revolutionizing the visual mobile experience. To help visually impaired users distinguish between different app icons, we assigned different bitmap images (consisting of vivid stripes, patterns, or hues) to each app icon on the screen. We then used TDap to create a tactile map of the screen, with each bitmap image mapping to a distinct tactile sensation. Ideally, app developers would be assigned these bitmap icons (like gravatars) with each app, and the tactile experience could simply be enabled by adjusting the OS settings.
To help you visualize this, see Screenshots 1-2, which display the actual bitmap image corresponding to each app icon, which can be made invisible via "Toggle Mode" in order to display the normal app icons, as would be displayed normally. Screenshots 3-4 demonstrate the same concept via button placement on a calculator corresponding to different bitmaps and tactile sensations.
Further extensions we considered included working with the Paypal Card IO recognition technology, in order to help users sense when the green lines surrounding their credit card are fully connected. We would do this by simply mapping friction to that specific shade of green, so users could trace out the rectangle with their finger. Additionally, we want to develop this app further to also serve hard of hearing users, by allowing them to assign distinct vibration ringtones to different users, much like hearing users can.