We were inspired by an elderly persona who shows early signs of Alzheimer's. Our goal was to build a prototype of an application that is able to interact socially with people. By detection of interest and moods the application will offer positive feedback that will bring the person in a better mood and is able to assist in reminiscence therapy, for example.
What it does
The application autolearns about mood and interests by offering various resources, audio, images, video, and registering the cognitive reaction (like/dislike) as well as physiological reactions through the use of wearables. This data is stored in a database and can be used in therapeutic sessions with caretakes but - as importantly - also at home as an entertaining way to interact with the application as an autonomous system or in a session with both the application and caretakers.
How we built it
The system is build by Java and is running on a Tomcat application server. The prototype uses only images to detect interest (cognitive reactions) and one FollowFlow wearable as a physiological mood detector. In order to learn from the reactions of the user, we use IBM Bluemix to classify the images into positive, neutral and negative categories. Categories together with cognitive and physiological reactions are stored in an in-memory database and can be used in future sessions for example to cheer people up (show pictures they really like..) of in reminiscence therapy (showing pictures that stimulate the mind and memory of the elderly).
Challenges we ran into
The use of IBM Bluemix was new and finding a programming language that permitted us easy access proved to be a challenge. Also interfacing the FollowFlow watch to read out both the raw heart data, as well as the interpreted mood data by a third parties API, was time-consuming.
Accomplishments that we're proud of
Having a working prototype that shows all aspects of our proposal.
What we learned
Again we learned about the huge challenge society faces with people getting more and more older. There is great need for technological support with the changing demographics and the costs involved. Technically we learned how to work with the IBM Bluemix API's.
What's next for CareTeam
- Adding an CareTaker section to upload specific resources to the personal workspace of the elderly user.
- More research on the correlation between congitive and physiologic response to the resources shown.
- Prepare the application for use on a low-costs mobile robot platform (such as Tiger or a Turtle-bot based platform) to add functionality to existing care-taking platforms.