Inspiration

We were inspired by real-life challenges faced by stroke and elderly patients who struggle to communicate their needs and safety risks. We wanted to create an AI-based system that ensures patients are never alone in emergencies, especially during critical falls or health deteriorations.

Problem Statement

Falling is one of the most common threats to hospital patients, especially those who are elderly, weak or confused. The U.S. Agency for Healthcare Research and Quality estimates that 700,000 to 1 million patients experience clinical harm due to in-hospital falls each year, and more than one-third of them result in injury, including bone fractures, head trauma and other potentially life-threatening injuries. Even among hospital patients who are not seriously injured from their falls, the psychological damage associated with fear can have lasting detrimental effects on their overall health.

Sources: https://www.boyerslaw.com/is-it-medical-malpractice-when-a-florida-hospital-patient-falls/ https://pmc.ncbi.nlm.nih.gov/articles/PMC6446937/

Use Case of Problem Statement 1 - Wrongful death of Mr. Mellon due to fall

Mr. Mellon had a history of falls and was a fall risk when he was admitted on May 30. He was seen in the Emergency Department, and at 7:38 a.m., he was assessed as a Level 2 fall risk, which is the highest-level fall risk a patient can be labeled. An assessment completed by a nurse at 10:13 a.m. again labeled Mr. Mellon as a Level 2 fall risk, and she began documentation that a bed alarm was being utilized as a fall intervention for Mr. Mellon’s safety.

Sources: https://medmal1.com/results/casefile/failure-to-set-bed-alarms-resulting-in-a-fall-and-death

Use Case of Problem Statement 2 - Injuries From Fall at Hospital Lead to Patient’s Death

Background: On Dec. 26, 2014, an 84-year-old man underwent a procedure for pacemaker implantation. Shortly before midnight on Jan. 2, 2015, the man was administered zolpidem to help him sleep while recuperating from the surgery. A few hours later, at approximately 1:30 a.m., a nurse moved the man to a chair beside his bed, as he was still unable to sleep. At some point that night, the man fell out of the chair and hit his head, suffering a laceration to his right ear that required stitches.

After the injury, a CT scan showed a subdural hematoma and, in the days following, the man’s brain swelled and bled. The hematoma caused brain tissue to push onto his spinal cord, rendering him paralyzed on the left side of his body.

Sources: https://www.reliasmedia.com/articles/143046-injuries-from-fall-at-hospital-lead-to-patients-death-3-million-verdict?utm_source=chatgpt.com

What is Sepsis?

Fever can be viewed as an adaptive response to infection. Temperature control in sepsis is aimed at preventing potential harms associated with high temperature (tachycardia, vasodilation, electrolyte and water loss) and therapeutic hypothermia may be aimed at slowing metabolic activities and protecting organs from inflammation. Although high fever (>39.5°C) control is usually performed in critically ill patients, available cohorts and randomized controlled trials do not support its use to improve sepsis prognosis. Finally, both spontaneous and therapeutic hypothermia are associated with poor outcomes in sepsis. Keywords: body temperature, fever, sepsis, septic shock, antipyretics, prognosis

Source: https://pmc.ncbi.nlm.nih.gov/articles/PMC10644266/

Use Case of Problem Statement 3 - Death of Savita Halappanavar due to Sepsis as a result of lack of monitoring on her temperature.

Her temperature at 9am on Friday 26th was 40 degrees and she had an elevated heart rate of 140 bpm. Savita continued to deteriorate, despite the administration of a range of supports and antibiotics. Sepsis was now engulfing her body.

Through the morning of Saturday 27th she remained critically ill. At about 4pm, she deteriorated suddenly and dramatically, becoming highly contusive and hypoxic – bruised in appearance due low oxygen saturation levels.

Through the day her condition became more extreme. Her heart rate was 150 to 155 bpm, her lactate levels were 24, grossly exceeding the desirable level of 1.5, and she was described as “very oedematous, fingers flexed tightly, feet extended and very stiff”.

At 10pm, alarms rang on Savita’s ventilator and she was given an intravenous infusion of cisatracurium, a drug to relax her skeletal muscles to help her breathe. At 11pm, she remained unresponsive. At midnight she was washed, and her bloodstained pads were changed. When the nurses tried to turn Savita, her heart rate increased from 140 bpm to 190 pm, so they stopped to let her heart rate settle.

The doctor saw me. She came to me and said, Do you know what is happening? Savita is dying Between 12.30am and 12.45am on Sunday 28th, as three ICU nurses sat by Savita’s bed, the monitor showed her heart rate becoming chaotic. Her pulse was not palpable. CPR was commenced by Dr Aoife Quinn, staff nurses Veronica Rafftery, Therese Connolly, Jeurgen Schone, Aine Nic an Bheath and Jacinta Gately.

After several minutes, nurse Gately ran to get Praveen, who was waiting with friends outside intensive care. She told him Savita’s heart had stopped, that they had commenced CPR. She asked him if he wanted to be present. He said he did. She took his hand and asked if he knew what was happening: that they were losing her.

Praveen recalled a “big team” around his wife. “They were pumping her heart,” he told this reporter in the weeks afterwards. “The doctor saw me. She came to me and said, ‘Do you know what is happening? Savita is dying.’ They were commencing compressions.”

CPR continued until after 1am on October 28th, when a decision was made to stop. Savita suffered a pulseless cardiac arrest. At 1.09am, Savita died.

Sources: https://www.irishtimes.com/ireland/social-affairs/2022/10/22/savita-halappanavar-10-years-after-her-death-will-irish-abortion-laws-be-reformed-further/#:~:text=Her%20temperature%20at%209am%20on,27th%20she%20remained%20critically%20ill.

What it does

CareBotix monitors patient health and safety using real-time environmental sensors (temperature, humidity, sound, and distance) along with AI-powered fall and posture detection through a camera. When a fall, critical movement, or unusual sensor reading is detected, the system automatically sends instant SMS alerts to designated caregivers using secure email-to-text messaging. All critical events are logged into a MongoDB database for tracking and future analysis.

How we built it

We built CareBotix with a modular architecture combining hardware and AI-powered software: Backend Server: Developed using Flask and Python to handle video processing, sensor fusion, and real-time alerts.

Computer Vision: YOLOv8 models detect falls, lying postures, and abnormal patient movements from live video streams.

Sensor Integration: Arduino sensors monitor temperature, humidity, distance, and sound levels for additional environmental awareness.

Alert System: We used SMTP (email-to-SMS gateways) to send instant emergency notifications directly to caregivers’ phones when critical events are detected.

Database Logging: MongoDB stores all patient event data, enabling future analysis, trend detection, and dashboard visualization.

Frontend Dashboard: A PyQt6-based real-time dashboard displays patient status, sensor readings, and video streams for quick monitoring.

Edge Processing: Local hardware (camera and sensors) processes data efficiently, minimizing latency and ensuring rapid response times.

Our system is designed to be low-cost, scalable, and easily adaptable for hospitals, nursing homes, and even home healthcare settings.

Challenges we ran into

Configuring live AI models like YOLOv8 to detect falls accurately with limited training data. Dealing with SMS service restrictions like Twilio A2P registration delays. Integrating Arduino serial data streams with Flask backend without lag. Managing asynchronous sensor reading, camera input, and alert triggering all together.

Accomplishments that we're proud of

Created a live demo where a fall triggers an automatic SMS alert in seconds. Successfully integrated multi-modal inputs (vision, sensors, speech) into one system. Designed a system that could scale for real hospitals and healthcare facilities. Built a real-time dashboard for caretakers to monitor multiple patients remotely.

What we learned

Real-world healthcare systems need redundancy (SMS + Email + Visual confirmation). Timing and real-time data management are critical when dealing with emergency alerts. AI models must be integrated carefully to avoid false positives or delayed responses. Managing cloud databases (MongoDB) and backend servers simultaneously.