The communicable disease COVID-19 has impacted all of our lives. However, one must not forget that the world still faces the issue of non communicable diseases, the dark horse which threatens to disastrously ruin our lives. COVID-19 has exacerbated existing issues, such as the mental wellbeing of employees and their decline in productivity, as more and more people work from home, by necessity, for the collective good of public health. With the balance between work and life being disrupted, people are now clamouring for the ability and the means to take charge of their lives and seize the day for themselves in the most productive way possible. Coupled with the recent focus on physical health in the age of sedentary lifestyles and diets which are too rich in calories and the twin evils of salt and sugar, there exists the need for a master app that is able to harness the power of machine learning and use it for the betterment of mankind. While there are apps that exist to boost productivity and physical wellbeing, there are no apps that integrate both - this is what has driven us to embark on this heroic endeavour.
We aim to use Machine Learning to integrate the tripartite relationship between productivity, diet and lifestyle, and achieve personalized productive health.
What it does
MLivin’ It is a phone app that aims to boost user productivity and health. It aggregates user data from multiple sources to provide a holistic Productive Health Score from a Health Score that measures physical and mental health, and a Productivity Score that measures daily productivity. Powered by multi-modal machine learning, the app analyzes data collected from digital wearables, app usage, meal photos and user input, over time, to build up an accurate profile of the user’s health and productivity. MLivin’ It identifies factors that boost or affect user productive health, and suggests actions that can be taken to raise the Productive Health Score.
MLivin’ It does not share data with external parties. User data will be encrypted and anonymised before being safely sent to the server for machine learning analysis.
How we built it
MLivin’ It was built in Adobe XD, with background code for the dietary image classification coded in Python using the pytorch library. You can find source code for our image classification model at the MLivin’ It Github repo, along with more details about the model itself and our app development process!
We employ a transfer learning approach to tackle the challenge of starting with a small input dataset, and achieve a very decent 93% accuracy with our vaildation set! Once classified, nutritional data is then pulled from databases such as HPB's FOCOS dataset, the 1972 East Asian Food Composition table as well as Nutritionix, the world's largest verified nutrition database.
Challenges we ran into
Due to the short nature of the hackathon, we weren’t able to build the full ML model that we envisioned since we lacked the comprehensive dataset that we required for our input. However, we’re glad to be able to provide a portion of our model through the development of a computer vision model! In our implementation, images of local foods were sourced from google images manually - this process can be further optimized by data mining images directly from the google images api to automate the dataset population process, and increase the efficacy of our model training to accommodate a much greater number of classes. We are extremely pleased to be able to showcase a working prototype of the app in time for this project submission.
Accomplishments that we're proud of
We successfully ran a preliminary test with the machine learning algorithm using a sample of 30 food images randomly sourced online and achieved ~90% accuracy identifying the correct food based on photos. We also developed an application (demo video) to provide a glimpse into the simple user interface of MLivin’ It and at the same time, showcasing a vision for novel data analysis output customisable for an optimal user experience.
What we learned
Creating MLivin’ It has raised our awareness of how powerful machine learning can be in harnessing readily available personal data to improve personal health and productivity. We have learned more about design thinking and how to successfully apply the different concepts in a practical manner to achieve a successful and desirable outcome. We have learned to innovate and harness each other’s skillsets (app creation, machine learning, etc.) to tackle our identified problem and construct a novel, working solution within the tight time frame.
In this day and age of social isolation where everyone is physically far apart from one another, we have experienced the challenges associated with mental wellbeing and productivity. Furthermore, due to increasing digitalisation and the pervasiveness of sedentary lifestyles, we have also realised the importance of physical health and how our health is linked to our work performance. These have enabled us to empathise with our target audience and learn about how we can design the app to suit their needs.
What's next for MLivin' It
We seek to improve the machine learning algorithm in calculating Health and Productivity Scores by altering the weightage assigned to specific parameters in different ‘day modes’. We also plan to incorporate more parameters from data that could be collected from the wearable devices/mobile phones in the algorithm to provide Smart Scores and Suggestions that are more nuanced and insightful. In addition, we hope to be able to integrate a greater number of nutritional databases into the model, to support a greater number of different cuisines beyond our current database of common and local Singaporean dishes! Finally, to ensure sustainability, we propose a subscription-based business model that allows subscribing users to access more in depth trend analysis of their Productive Health on a monthly and yearly basis.