After four years of attending hackathons, I realized I never tried a project for disabilities, and I decided to give it a try this time. I also started to drive more often this year after getting my own car, which encouraged me to do a driving-related project. I believe very few people with visual impairment will know what the driving experience is like, and some people will never get to know the experience. Therefore, I decided to create a platform to provide a unique, driving experience to people with visual impairment. -Ben
What it does
Project Podori is an interactive application for the visually impaired to experience driving a car through providing the driving atmosphere. There are three parts in this application: simulation software, steering wheel, and pedals. The simulation software visualizes the street with cars in multiple lanes and is purposed for spectators to see how the user is performing. It demonstrates the movement of the car and its speed based on user inputs. It also provides user instructions ranging from how to operate the car and important driving practices to keep in mind through sound and touch sense. The steering wheel is the controller to move the car in the simulation software. We built our custom steering wheel to provide haptic feedbacks to the user and to detect the movement of the steering wheel. There are vibrating motors on the handle to provide feedbacks to the user when other cars are approaching from the side. There are also lane signal bar for signaling lane changing while driving. There are two pedals: driving and braking pedal for moving and stopping the car in the simulation software. Two pedals use compression springs to provide physical sense to the user. Based on which pedal is pressed, the car in the simulation software will move or stop.
How we built it
We integrated the computer software with a wide variety of hardware components. For the steering wheel, we used a baby toy steering wheel and attached an accelerometer to determine the position of the wheel. An IR sensor with Arduino Uno is attached on the side to detect the position of lane changing signal bar. There are two two vibrating motors on the handle to provide haptic feedback to the user. On two pedals, we used compression springs and IR sensors with Arduino Uno to detect which pedal was pressed, left (brake pedal) or right (accelerating pedal). The simulation software is a multi-thread python application built using pygame. By using pygame library, we created 2D map of streets with cars going on both sides. After reading values from the arduino, the car model in the application either accelerates or rotates. We created step-by-step instructions for driving through the combination of tts (text to speech) audio generation and our own simulation logic for detecting if the user did the right action. Through software, we were able to detect collisions of the car objects and notify the user of the result. This application was integrated with the arduinos through our serial communication protocols.
Challenges we ran into
We had a difficulty building our custom steering wheel and pedals to detect the user’s movement. We had trouble figuring out how to design our hardware components so that the hardware input measurements accurately translates to what happens in our simulation. For example, we weren’t sure where exactly to place the accelerometer on the wheel, and how we’re detecting if the user is stepping on the pedals or adjusting the signal handle. We went through different iterations of placing sensors in different places and packaging our hardware until we got pretty precise input measurements from our hardware that accurately translated to what’s supposed to happen in our simulation.
We also coordinated the serial communication between the arduino and python application. We tried to send only essential information using the minimum size of bytes to maximize the communication speed. We created two extra threads in the python application, responsible for both receiving and sending data to/from each arduino. The python application also required lot of time to learn how to use pygame library to draw models on the screen and render the screen. We created our own series of logic to come up with comprehensive instructions on driving.
Accomplishments that we're proud of
We were surprised that our custom designed steering wheel and pedals integrated with our python application successfully. Receiving and sending data between our python application and two arduinos did not impact the speed of the simulation software. We did not have a prior experience building GUI applications on python using Pygames and successfully created a working application. We were able to create both software and hardware components purposed to provide interactive driving experience to the people with visual impairment.
What we learned
We learned a lot about multithreading, creating robust Arduino code, and designing a hardware project (cardboard boxes are actually quite useful). What's next? We would like to add more types of street, such as intersection streets and three way streets. We also want to add more components to the street, such as traffic lights, stop signs, and crosswalk for more realistic experience.