Iago for Dementia Rehabilitation

A rehabilitative software/hardware project to help Dementia patients remember their loved ones.

Comment

Inspiration

Our elderly are often the first demographic neglected by the advances of technology. However, they are also the group that stands to benefit most from new accessibility features and processes that allow an unprecedented ease of access to devices, like smart phones and home IoT networks in ways previously thought to be unattainable for that demographic. Features such as text-to-speech, magnifiers, and voice control all make devices that were previously thought to be too complicated for the elderly simpler to use than ever before. However, instead of adding features that allowed the elderly to conform to generic device usage, we wanted to create something that was tailored to the needs of dementia patients, a disease that affects one in fourteen people over the age of 65. While there is currently no cure for dementia, we wanted to create a technology that would help patients maintain connections with their loved ones and give doctors’ a better look at the progress of their patients to increase the quality of life of these patients.

What it does

Iago combines hardware designed for accessibility with a web application to serve two main functions: a tool for patient memory exercise and a portal for doctors to easily monitor patient progress and reconnect patients with any loved ones they may need more assistance with remembering.

Patients can log in through our patient portal and start the exercise. A 2x2 grid of images will be shown on the screen along with a name. The patient can then start to match the name with the correct face using the hardware button panel. The button panel and the photo grid are aligned in shape and color to help the elderly patient match the button with the corresponding image. If they correctly match the name to the corresponding face, then the patient gets a message that it is correct. If they get it wrong however, they get an error message that tells them who is actually in the picture. In both cases, the data of which faces were guessed correctly versus incorrectly is stored for future reference. After the user finishes the exercise, they have the option to replay or conclude their exercise.

Doctors can log in through the doctor portal and monitor their patients’ progress in identifying their loved ones. On the left panel as shown in the above screenshots, they can view the names of the patients that they oversee. If a doctor clicks on a specific patient, they can view specific details about the patient, including basic information such as name and age, as well as their frequency of using Iago. Importantly, there is a section in the doctor’s portal for each patient that displays an analysis of who the patient remembers very well as compared to the people they have trouble remembering. For the people that the patients have trouble remembering, the doctor has the option to automatically send an email to a close family member to remind them to check in on their loved ones. Lastly, the statistics of the elderly patient’s progress are displayed in a graphical format at the bottom.

How we built it

Hardware Component

We created and 3D printed a CAD design for the enclosure of a four button circuit connected to a Raspberry Pi that uses the GPIO of the Pi and Python Flask to send button presses to a web application for processing. Accessibility is at the forefront of our physical design, so the buttons are large with embossed shapes for those who are visually impaired or color blind.

Software Component

We created a web application using a JavaScript and React framework with Node packages which connects to the Python Flask hardware side via HTTP. Visitors to the web application first see a splash screen where they can learn about Iago. The user can then log into the portal in their role of either patient or doctor.

Patient Portal

Once a patient has logged in, the patient portal includes software that directly connects the web application to the physical buttons to execute the memory exercise. The exercise was created with a mixture of React, JavaScript, and Bootstrap. In order to facilitate the communication between the raspberry pi and the web application, we created a python flask server and communicated via HTTP requests. When we detected a button press on the hardware component, we sent a POST request back to the flask server to signal that we detected and processed the press, and reset the button press. The web application polls approximately every second and sends a GET request to the server. If the button value has been updated, the GET request returns a valid button value and the web application responds with a success or failure message (depending on whether the button pressed was correct.) With each success, the patient’s score increments by one. After five guesses, the game concludes and the web application displays the user’s final score and allows for the option to play again or navigate through the website.

Doctor Portal

Once a doctor has logged into the portal, they see a live updated chart of a selected patient’s information and progress. The web application displays the patient’s statistics using Nivo graphs reporting the overall performance of the patient and which people they seemed to remember the most. For the purposes of the hackathon, we mocked the data, but if we were to further implement this, we would query a database upon refresh in order to get graph data. Moreover, if a patient was having particular trouble remembering a particular person, we gave the doctor the option to alert the person to check in on their loved one using an automated emailing system. We used EmailJS in order to automate the emailing process. In the sidebar, we added a list of patients so that the doctor can toggle and manage each of the patient’s relevant information.

Challenges we ran into

  • CORS errors galore
  • Challenges connecting the Raspberry pi to the wifi network
  • How to automate emails?
  • What are react states?
  • Scoping and defining the problem space

Accomplishments that we're proud of

  • Figuring out how to host a web application at a reliable web address using ngrok!
  • Setting up proxy servers to bypass CORS errors
  • Figuring out React States
  • Collaborating and building a project in a virtual hackathon!

What we learned

  • How to prototype and build a product within 24 hours
  • Defining and exploring the problem space
  • How to work together on a larger scale product
  • How to communicate between hardware and software

What's next for Iago

We are interested in performing further data analysis on Iago data. We want to make it possible to perform more methods of data analysis in order to provide further insight to family and doctors. Moreover, we want to develop a family portal, so that families can upload more pictures to facilitate the treatment of the patient. Moreover, through the family portal, family members can view and track the progress of their loved one. Lastly, we want to make sure that it can be easily deployable to seniors around the country. We plan on doing customer research to make sure the deployment process is as painless as possible.

Share this project:

Updates