Inspiration
We are a group of students passionate about the intersection of health and artificial intelligence. We saw that there was generally a lack of machine learning used in the health field compared to other fields despite the potentially large impact it could have on the wellbeing of other's. We thought that creating a webapp in the health field would allow us to create the most impact by directly helping people live a more healthy lifestyle.
What it does
KnewHealth is a health tracking webapp that provides features that aim to help users live a healthier lifestyle by utilizing deep learning. One of the main features on the webapp that accomplishes this is a calorie estimator that is able to predict how many calories a particular food is based on an image uploaded by the user. In addition, the feature would allow users to also see the nutritional breakdown of the food by providing an estimate of the amount of carbohydrates, proteins, and fats. Also, this feature would output whether the food in the image is considered either a healthy or unhealthy food.
How we built it
The webapp was built using flask. Login information for users and food image data are stored in tables in a CockroachDB database. The model used for the calorie estimator was built using a pre-trained vision transformer on 100+ food classes.
Challenges we ran into
There were a few challenges we faced throughout the course of building this project. One of the biggest challenges was training the model used in the calorie estimator. We had originally planned to utilize transfer learning with the ResNet-50 model to make predictions for the calorie estimator, however, we ran into many technical issues with this that we thought could not be fixed within a reasonable amount of time. Due to this, we changed our approach to using a pre-trained vision transformer instead which circumvented many of the issues we were facing.
Accomplishments that we're proud of
We are very proud of building a fully functioning webapp that is able to take in a food image and output the calorie, carbohydrate, protein, and fat estimates in the food in a very short amount of time. The webapp even allows users to create an account with login with a username and password which the user can use to login to use the calorie estimator.
What we learned
We learned alot about using several technologies such as flask, pytorch, and CockroachDB. Since much of the team did not have much experience with those technologies, it was a challenging but rewarding experience learning how to use them in a short period of time.
What's next for KnewHealth
We plan to implement more features such as a pushup and squat counter using video to help users keep track of how many of a certain exercise users are doing. In addition, this would ensure the user is doing the exercise correctly as it would only count the exercise if it is done with proper form. With these additional features, we hope to continue working towards our goal of allowing people to live a more healthy lifestyle.
Log in or sign up for Devpost to join the conversation.