Over half of car accidents relating to medical conditions is due to a medical condition involving a change in heart rates. Car manufacturers have been trying to prevent human related accidents lately through technologies such as Computer Vision and Machine Learning such as Audi and BMW cars detecting for sleepy drivers to Tesla cars detecting if a driver is unresponsive at the wheel; however, none of these manufacturers utilize embedded tech to measure and monitor essential vital signs such as your heart rate bpm or ekg rating. With the advent of more emerging technologies such as Augmented, Virtual, and Mixed Reality, it is essential to provide essential information to a driver at need whether it be through the windshield of the car or through the drivers eye wearable tech.

What it does

A steering wheel with two pulse sensors records the users heart rate data at fixed intervals and feeds it through a trained Recurrent Neural Network machine learning model to perform anomaly detection to detect whether there is any sort of medical emergency happening as related to the drivers heart rate. If there is any anomaly detected, the user is warned through their windshield or AR glasses of their current vitals and instructs the car to slow down and come down to a stop to the side of the road. At its essence, much of the stack is a proof of concept due to not having essential technologies.

How I built it

Two pulse sensors embedded on a 3-D printed steering wheel is connected to an arduino board which contains several LEDs. The LEDs are used to signify whether the car needs to slow down or not. -insert stuff about AR- An LSTM Recurrent Neural Network model based on an Encoder Decoder architecture is trained with time series data (utilizing the PyTorch framework and Google Cloud Deep Learning VM to train via a Nvidia Tesla K80 GPU) such as ECG signals and Heart Rate data (from our sensors) to learn and predict the behaviour of the data. The prediction errors within the model are used to compute further statistics such as the mean and covariance which are used in combination with the predicted time series and predicted error to generate an anomaly score which is calculated in an anomaly detection script. The anomaly detection script provides essential metrics such as the f1 score (the measure of a tests accuracy), precision (positive predictive value), and recall (sensitivity). Anomaly scores that spike up signal a trigger for the car to slow down.

Challenges I ran into

There is a huge lack of AR/VR development on data visualization so there is little to no libraries or tutorials on this. Having to use AR through using Javascript and the web. BPM data is very sensitive and prone to mistakes and not as good as a measure as ECG/EKG; however, there are no devices available at the makeathon to record that. Training large amounts of time series data on a deep (as opposed to shallow network) even on a GPU is very time consuming which leads to not enough development time to deploy the model. 3-D printing is time consuming

Accomplishments that I'm proud of

We managed to take an idea and bring it to reality through the use of arduino and supplied sensors having little to no arduino experience prior.

What I learned

We learned how to save an arduino's serial reading onto a database and to allow that database to be queried anywhere using Serveo.

What's next for Save Boi

Update the Machine Learning architecture to a LSTM Autoencoder model that utilizes the reconstruction error for anomaly detection as opposed to prediction error since there are a few papers with promising results. Being able to interface more vitals.

Share this project: