Inspiration
We drew our inspiration from the movie Big Hero 6 character Baymax who was a personal health care taker robot. Instead of a robot, we tried to create a web portal where we could try and do similar things.
What it does
Currently, what it does is that it can track emotions on the face and can help detect if a person is happy or sad or psychologically low using the computer webcam. It then provides continuous analysis based on the person's facial expressions. This feature is coupled with a search of his/her social account which can give more idea about his/her recent behaviour on social media as it can surely provide some indication as to how is the person behaving psychologically right now!
How we built it
Microsoft Cognitive APIs run at the core of the application. We used the Emotion API to track facial emotions. Ideally, an automated agent will be talking to the person and will be using computer vision to keep track of the emotions. Right now, we are using the laptop webcam to accomplish this. We could have the user watch some video suggested by a doctor or just track the user behaviour while he/she is surfing. The web portal takes snapshots of the user at regular intervals and tracks in real time what are his/her feelings which are shown with the help of a bar graph. Along with that, a separate module looks at the tweets which the user has made in the recent past which could tells us more about how exactly has he been feeling recently. We are using tweepy Python module to fetch the tweets using user's Twitter handle and are then using Text Analytics API to analyze the fetched tweets by calculating sentiment for each of the fetched tweet.
Challenges we ran into
One of the challenges we faced was to get the continuous video tracking and the real time graph updates thing getting to work.
Accomplishments that we're proud of
Since it was our very first hackathon, we did not have much clue as to how we were going to go about everything and looking finally at what we have made, can be very happy with what we have come up with. Obviously, there is a lot of scope for improvement and now we know exactly what goes about in a hackathon which will surely help us be more prepared the next time we participate.
What we learned
As I just mentioned, we learned quite a lot as far as participating in a Hackathon is concerned. Another thing which we learned was working on front end which we have always stayed away from. Again, there's a lot of improvement to be done but I think, we can say that it was a great learning curve for us.
What's next for Health Portfolio Manager
Our inspiration - Baymax is a great example of how an ideal personal health care taker should be and if you compare our portal with it, there are a lot of areas to work on. Firstly, we would like to use something like Google Fit API to try and fetch user health information which can be a great source of information for an automated agent like this. Another thing which we would like is to make a Speech recognition and analyzer system to recognize and analyze user psychological behaviour from the usual conversations which the user has in his day-to-day life. There's still a lot of combined analysis to be done on all kinds of data which we have obtained. Yes, still a long way to go but if we could get all the things that we are looking for here, then nobody can stop us from creating a real life Baymax and who knows, it could even turn out to be better than Baymax itself!
Log in or sign up for Devpost to join the conversation.