Vehicles have come a long way since they have started, as they become more efficient and technologically equipped. But driving on highways there's a lack of communication between vehicles. Such a communication channel between vehicles can help convey important messages such as emergency vehicles approaching or cars parked on the side of the road. The ability to push such messages through from one driver to another can help avoid accidents and improve the safety of our roads.

What it does

A user defined builtin message is sent from one car to another via LED headlights/taillights. The message receiving car reads the message using its camera, which then uses machine learning to understand and display the message to the driver.

How we built it

We created a simulation of this communication channel by using LED headlights/brakelights and rear/front view cameras to send and receive data. An LED strip, connected to an Arduino, displays a message dependent on a serial input being received from a raspberry pi. This light pattern follows a communication protocol we have set up from scratch. An image is then captured by a camera connected to another raspberry pi. This photo is sent to a server with larger processing power for image processing, were the image is first blurred then an HSV mask is applied. Then we run a color filter and blacken everything else in the image. Using gradient analysis to find the centers of the LEDs where we draw a circle which would represent an on LED. After the message is extracted from the image, a symbol and explanation are displayed to the driver representing an emergency or status signal.

Challenges we ran into

Understanding efficiently manipulating masks in openCv Initial algorithm misidentifying colors when trying to define objects. Improving image quality for better object detection. Processing limitations of RaspberryPi Wireless network frequently disconnected interrupting SSH and cloud processes

Accomplishments that we're proud of

We have successfully implemented the protocol we designed after a lot of difficulties when it comes to machine learning and decoding the message. We were able to solve a common problem and hazard issues people on the road face daily.

What we learned

We learned a lot about communication protocols, and how to have a successful light communication by reading some articles. We also learned how to use openCv which is the bases for the machine learning LED strip identification. It was hard to filter out other light sources and reflections, but with enough masks it was feasible.

What's next for CarTalk

Implement the image processes in a more powerful computer to be able to increase the read rate until the frequency is high enough to look invisible to the eye due to the fast refresh rate.

Share this project: