Pixie Rover

An AI-powered IoT robot that monitors for patient's safety and well-being.

Comment

👋 Intros

Hi! This is Jordan, writing on behalf of this project, alongside my teammates, Jenny and Ray. We're excited to participate in Cerebral Beach Hacks as our first time and create our prototype version of Pixie Rover!

💡Inspiration

I was inspired to create a low-cost and scalable solution for hospitals and labs in monitoring patient's safety while abiding by the Health Insurance Portability and Accountability Act (HIPAA). To provide a personal story, my now-late grandfather's health was consistently under-monitored and neglected due to low hospital staff, often lending himself to become sicker than deemed necessary. The Pixie Rover is intended to be a long-term solution that aids hospital operations in wake of nursing staffage shortage, while providing timely and humanizing care for patients. In this prototype, it will specifically look for PPE equipment to ensure compliance with patient safety protocols, but this product can expand in both scope and features depending on specific healthcare needs.

🎯 What It Does

The device will identify PPE worn by medical personnel to ensure compliance with patient safety protocols. The current operation of the Pixie Rover is as follows:

  • An accessible webpage runs the Flask server on the robot.
  • The camera is turned on and records a video.
  • The video is uploaded to Twelve Labs following the prompt ("Is this person wearing PPE equipment?")
  • It generates a conclusion based on the prompt.
  • The robot will respond based on this conclusion, nodding to say that the person is in compliance or shaking its head to say that they are not in compliance.

⚙️ Build Details

  • Jordan: Designed, 3D-printed, and assembled the body of the robot utilizing modular pieces.
  • Jenny: Connected hardware pieces (micro-controller, Raspberry Pi, and external camera).
  • Ray: Developed the Flask back-end to facilitate communication between the Raspberry Pi and the AI model, orchestrating network requests through the Twelve Labs API.

❓Challenges

We all underwent multiple challenges that resulted in huge-learnings for us during this hackathon. Jenny had to learn how to control the micro-controller with the Raspberry Pi with little coding experience. Ray had to learn how to use the Twelve Labs Generative API from scratch. Jordan had to ensure the product was properly modular within the limited time-span, while ensuring the 3D-printed parts fitted together.

👍 Accomplishments

We are very proud of the hardware we designed and executed within the 24 hours of hacking, resulting in an MVP with a viable medical use case. We were especially thrilled to see that the final design of the robot once it was 3D-printed and see that it was receiving positive reactions from other hackers who noted the human touch of Pixie Rover. In addition, it working well with Raspberry Pi and saving the videos properly, once it responded to the image was another great accomplishment.

📝 Lessons

We learned how easy it was for Twelve Labs to understand our prompt (our test example is checking if the person is wearing PPE) after viewing our video. Although the processing time could be improved, we are satisfied that our robot is able to take the information from the video and nod or shake its head and can see many viable use cases that can scale accordingly with real-world healthcare problems.

🌐 The Future

Our next goal is that the robot will be able to drive itself and move between rooms. It is already built ahead with room for wheels, in intent of creating a modular design, but due to time and lack of materials, we were not able to include them.

🕺Most Innovative Product (Sponsored by PMA)

We envision the Pixie Rover as a flexible solution that accommodates a range of budgets and healthcare needs, thanks to its low-cost production and cloud-based technology. By leveraging data collected by Pixie Rover, healthcare providers can enhance ongoing care in response to nursing staffing shortages and the rising demands expected over the next fifty years, particularly in a post-pandemic landscape.

Pixie Rover has the potential to accurately monitor medication administration, detect accidental spills, track patients' eating habits, and much more. It can be tailored to meet the specific needs of hospitals requiring mobile surveillance and 24 hour monitoring, offering a cost-effective alternative to high-end surveillance cameras, while delivering real-time data that supports healthcare providers.

In comparison to competitors like Aerion's The TUG, Pixie Rover can be deployed in fleets that adapt easily to changes in floor plans, remaining both affordable and efficient with minimal IT maintenance required. Because they are small, that means they can easily be handled by staff and poise no threat to patients. This makes Pixie Rover not only an innovative solution but also a realistic one, poised to drive meaningful improvements in healthcare delivery without imposing additional burdens on healthcare payers, hospital staff, or patients.

⚖️ Responsible AI First Challenge

When considering the future of AI in healthcare, we took several important factors into account. First and foremost, we prioritized patient privacy, specifically HIPAA laws within the US. By utilizing Twelve Labs' AI cloud solution, we ensure that videos can be easily deleted in this prototype and would ideally be accessible only by healthcare providers.

Next, our goal was to create a face to an AI, which manifested as the Pixie Rover that seeks to empower human caregivers, while addressing the ongoing nursing shortage and increasing hospital demands. By designing Pixie Rover that can nod and shake in response to images, we humanize the technology. This approach helps maintain human oversight, alleviating concerns about AI and robots that may arise from public perceptions.

Lastly, we recognize that healthcare often reflects implicit biases that can adversely affect minority groups, including Black patients, who historically experience higher rates of negligence and mortality. Our product aims to combat these disparities by focusing on objective, realistic solutions that challenge healthcare discrimination and provide equity care.

Built With

Share this project:

Updates