Healthy Habits.ai

Inspiration

As college students, all four of us shared the concern that our sedentary lifestyle, which was magnified by the lockdown would lead to bad habits that affect us long term. In particular, we were concerned that us as well as many of our peers would often slouch and sit back instead of sitting up straight. In addition, with the pandemic, we were aware that many people were afraid of catching Covid. We realized that one of the most common avenues of transmission was having one’s hand in close proximity with their face. As Computer Science and Data Science majors, all of us recognized that technology could help address our concerns and because we were all interested in machine learning, we figured that implementing a machine learning algorithm to track one’s hand and one’s posture would be a great way to further our knowledge and address our concerns.

What it does

Healthy Habits.ai connects to your webcam and takes a picture of you every minute. You can either choose an algorithm that checks your hands to make sure they are away from your face or checks your posture to make sure that you are not slouching. If you are either having your hands close to your face or slouching, depending on the option you chose, the web server will give you a vocal warning as well as keep track of the total warnings it has given you so far.

How we built it

We started off with the machine learning algorithms where we found templates off of Github. After understanding what they did, we decided to use CVLib for the framework behind the algorithms. The good news was that it already came with the training data so we did not have to provide our own, especially for the posture detection part. However, for the hand detection part, we did have to take some selfies with our hands in a variety of positions. After finishing the machine learning algorithms we ran our server on Heroku to give it accessibility. The next part was frontend where we used HTML to make a website that would display our model for visitors. We also implemented buttons which we managed via Javascript. For our counter, we represented with a Javascript variable which we would display through HTML.

Challenges we ran into

Overall, a challenge we faced from the start was our inexperience in developing a web application. However, through the help of our mentor Anish and our eagerness to learn, we quickly beat this obstacle and were able to accomplish our initial goal. Furthermore, with everything virtual and all members based in different timezones, collaboration was a lot more challenging than in person. However, we didn’t let this slow our progress and we were still able to meet regularly to maintain this essential component of collaboration.

On the technical side, we had issues with our face-hand detection. Originally, we had trained a yolov4 model with a custom dataset of 1500 hand and face images. Although the results of this training was wonderful (mean average precision of almost 98%), we could not use this approach because deploying an ML model that required a GPU to run properly was too tricky for us to set up.

Furthermore, a challenge we ran into was the struggle of deploying our application to Heroku. While deploying our web application, we ran into a memory issue as the size of the Tensorflow package was too large for Heroku to process, which slowed down our web application extremely. In addition, we also encountered a memory leak, so we spent a lot of time trying to find the cause. Because of these challenges with Heroku, our web application can only run locally.

Accomplishments that we're proud of

We are proud of building a web server and implementing machine learning algorithms all in our first hackathon. In particular, many of us were new to Javascript and HTML so we are surprised at how quick we picked up the new language. In addition, having never worked on a project as complex, we are proud of how we managed the numerous branches of this project. Finally, as 4 individuals from all over the nation that haven’t known each other prior, we are proud of how we gelled as a team.

What we learned

We are extremely proud that we were able to develop a web application with no prior experience. Furthermore, we also learned how to implement flask and deploy Heroku. Finally, as first-time participants of a hackathon, we learned how to collaborate as a team to build a project from start to finish, and jump over the hurdle of doing everything virtually.

What's next for Healthy Habits.ai

We hope to be able to deploy the app through Heroku and further improve the accuracy of the detection models. In addition, we hope to improve our face-hand detection by utilizing the yolov4 model with a custom dataset of 1500 hand and face images.