Inspiration
We were inspired by the lag in communication between nursing home patients and their caregivers on each individual patient's status, leading to an inefficient patient care system and ultimately a dangerous situation for those who may need immediate assistance. From this, we sought to develop a real time web application that detects and extracts patient emotion information with the power of AI to expedite patient status to nurses, hence the creation of Emotion Stream.
What it does
Emotion Stream is a real time web application dedicated to the jobs of nurses to retrieve and view patient emotional status. Most significantly, by integrating the Hume API for human expression, Emotion Stream combines computer vision in its live video stream of patients and the Hume AI model to display the top negative emotions detected from patient stream. By allowing for an accessible platform where nurses can view patient status and receive alerts when emotional stress is detected in specific patients, Emotional Stream seeks to enhance patient caregiver communication.
How we built it
We used React to build the frontend and Python, Flask for the backend. For the database, we used MongoDB to store patient, nurse, and report information. Most importantly, Emotion Stream centers around the use of the Hume API to analyze the expressions of patients to categorize into negative emotions.
Challenges we ran into
We spent time constructing the base framework of the application and learning to integrate the Hume API into the patient page. Connecting the Python backend and the React frontend to display patient info for the nurse page was also an area of implementation which took time. We also ran into some challenges in correctly configuring and sending alerts to nurses when a patient reports an unstable emotional status.
Accomplishments that we're proud of
We are really proud of the functionality in integrating the Hume API to detect in real time the top negative emotions that a specific patient is experiencing, as well as an efficient frontend backend integration to display an active notification system for nurses to log the statuses of all patients.
What we learned
Through the process, we learned how to integrate the Hume API into web applications and establishing connection between our React frontend and Python backend, including configuration of our MongoDB database.
What's next for Emotion Stream
For the future, we want to fine tune the Hume AI model for human expression with a custom dataset of elderly people to improve the accuracy of emotions detected tailored to the use of our application. We also plan to add to the model more nuanced emotions than those supplied to us by Hume AI. We hope to make the application more customizable for different health practitioners and more generalized to different patients' needs.
Log in or sign up for Devpost to join the conversation.