Single engagement to collect data
There are lots of apps for tracking our daily life, and people use them to become healthier. However, most of them requires manual input, and users start to skip recording in few days and totally forget about it in few weeks. 'Water the Onion' is a system which is specifically designed for workers who spend 1/3 of their day in office. It will automatically track hydration of users to make them healthier.
What it does
When users enter their name and drink water for 5 seconds, the system will be able to recognize whether user is drinking water or not. By following their behaviour with webcam, system can easily track how many times they drank water.
How I built it
WPF and Microsoft Custom Vision were used to build the system. Custom Vision model is completed with single engagement; entering name and drinking water for 5 seconds. It tags pictures with negative or drinking, and the system uses published model for prediction. Considering the system is going to be used in office, no more engagement was required: background and webcam angle barely changes in office.
Challenges I ran into
- Lack of time: Since this is new technology and I couldn't find team member to work on this project, I did not have enough time. Therefore, I implemented the functionalities with higher priority.
- Unexpected error: I adopted parts of code from Microsoft documentation and changed a little bit. Then I got bugs that prediction is not happening, and I couldn't get proper response from debugging. So I compared with project web page, then I found out that I was sending too many pictures in short time (there was a message from web). So I changed it to predict only once in 3 seconds to make system work.
- Privacy issue: Even though the image is directly sent to Azure, it might make some users worried. Therefore, I made the video not being visible in the app and did not train further after model is completed.
Accomplishments that I'm proud of
I built something, and I can use it in real life from next Monday.
What I learned
Basic idea of machine learning.
What's next for Water the Onion
Below are the items ordered in priority to make this system a little bit better.
- Make code cleaner: I was in hurry, so code is very ugly.
- Consider other factors in scenario; cup is changed, user wears different clothes, users pick up other objects, user does not get detected by webcam in engagement stage etc.
- Check whether I can export model and adopt in code: I'm not sure whether it is possible or not, but it would be good to check.