MakeTheLeap

• 

  Hand Gestures

When examining patients, medical professionals often find themselves in situations where their hands are not well suited to manipulating a computer’s trackpad or mouse.

Such situations often arise when they are holding items such as surgical instruments, when their hands have been exposed to contaminated materials or bodily matter from the patient, or if they are wearing latex gloves which inhibit the ability of the trackpad to detect the touch of skin. In spite of these situations, the medical field’s increased integration with technology renders interacting with computers necessary during these situations. In order to provide a way for medical professionals to interact with their computers even when their hands are not suited to contact devices directly, we developed a healthcare system that can be accessed hands free fashion using LEAP MOTION technology. We leveraged the LEAP Motion API to track finger movement and operationalized the corresponding movements using the pynput library which translated the finger movements into instructions for controlling the on cursor on the screen. Our motion tracking capabilities allow users to hover two fingers comfortably above the sensor to control cursor movement. Bringing the two fingers together instructs the cursor to click. Meanwhile we also developed two gestures. Performing a pinching gesture results in an on screen zoom while tilting of the hand up and down results in scrolling the screen up and down accordingly.

Meanwhile, our web application was created using Angular 6 and was designed to allow medical professionals to access patient data (stored in a Firebase database) such as diagnoses, symptoms, and imaging on the fly during an examination while being easily manipulated motion tracking of the fingers. The design of the application kept minimalism and accessibility in mind specifically to reduce the need for precision with regards to hand gestures.

Built With

• angular6
• firebase
• leap-motion
• python