heARt

• 

• 

• 

• 

• 

• 

• 

• This image is currently processing. Please be patient...
  

HeARt Health AR improves clinical data visualization & communication of the medical staff for efficient patient care.

According to a 2018 poll by Stanford Medicine, most doctors agree that EHR/EMR (electronic health/medical records), greatly contributes to physician burnout and needs a complete overhaul. Most problems in healthcare derive from poor communication and connectivity. The smart use of technology can help. HeARt is an AR application to improve clinical data visualization and communication, for a safer and more efficient patient care.

HeARth combines – Face recognition and patient identification – Access of EHR/EMR (electronic medical records) – Vital sign detection (heart rate via camera and machine learning) – Accessing data with hand gestures – Listening to the patient summary with audio – Recording of notes via voice and microphone

What it does

The heARt application offers face recognition with the built-in camera of the Magic Leap to identify the patient and access the clinical data. It also offers detection of vital signs such as heart rate via built-in camera and machine learning right at the patient. The doctor is then able to pull up medical records and listen to the patient summary with integrated audio files while with the patient.

Currently without heARt, over 60% devoted to each patient are being spent by doctors separately with the medical records, and an additional about 20% of time talking to the medical staff. The heARt application allows the doctor to access the medical records and recording of the medical staff while with the patient and to spend over 80% of the time with the patient.

The use of heARt lessens the total hours of work for doctors, adds valuable time at the patient for care, strengthens the doctor-patient relationships and effectively improves inpatient care.

The AR application can also connect with other sensors via Bluetooth to display vital signs and clinical to the doctor.

In addition to the easier work flow, it also offers a more hygienic solution by using hand gestures without touch of any screens of documents, as well as audio to listen to medical staff feedback and voice to record the diagnosis of the doctor.

How we built it

Languages: C#, Python Apis: Magic Leap API Hardware: Magic Leap, Windows laptop

Challenges we ran into

Networking the devices...

What's next for heARt

First steps made to use Magic Leap camera with machine learning to detect heart rate and other vital signs, and for face recognition to identify patients. Further establishing this. Also to add other sensors to integrate multiple vital signs and patient data into the AR application. Improve networking between computers and Magic Leap. Refine UI and UX. Already great feedback by mentors (accomplished surgeon and physician).

Our team

Hannes Bend: Founder of breathing.ai to personalize screens with biometrics and machine learning, multiple award-winning VR/AR projects with US-research universities and US-county on health, science, art and mindfulness.

Alaa Algargoosh: Ph.D. Candidate in Architecture | Building Technology & Architectural Acoustics at The University of Michigan | Master of Science in Architecture, Sustainable Architecture at University of Dammam, Saudi Arabia.

Nirali Oza: As a tech-savvy medical graduate and an altruistic designer, I am passionate about utilizing data, technology, and design to tackle the complex challenges facing today’s world. Bachelor of Medicine and Bachelor of Surgery, MBBS from the Government Medical College Bhavnagar, Gujarat, India

Matthew Allen Fisher WebXR Developer, 3D Modeling Enthusiast, Machine Learning Student, and CEO/Co-founder of Component Entity.

Tushar Purang Delhi-based engineer, AR/VR Developer, hackathon enthusiast with a passion for designing and building experimental user interfaces.