PRAP - Physical recognition activated prosthesis

The team (Asif not pictured)

Inspiration

We wanted to provide assistance to the disabled community through tech. More specifically, amputees have limited technologies which can assist them. Based on our previous work and experience, we know that modern prosthetics can often help but have much room for improvement. Our alternate solution provides a significantly major improvement to current actuation methods used in modern prosthetics.

What it does

Similar to most modern day arm prosthetics, the device mainly consists of a robotic hand. However, our project specifically also has a component which detects objects in real-time. This detection allows for the user to choose certain hand-held objects, and when detected, the hand will automatically grip its fingers around said object. This removes the need for manual actuation (in for form of buttons or even EMG readings), as the system will automatically close on its own.

Furthermore we have developed a mobile app which detects the grip strength of each actuation over time and collects data on each trial. Statistical analysis is then run on this data for optimal grip strength needed based on each individual user's habits.

How we built it

The main hardware prototype was developed by the segmentation and assembly of local craft wood. The fingers are connected via strings, and the grip actuation is initiated by 5 servo motors pulling each finger (allocated using Arduino). The object recognition system was developed in python by utilizing YOLOv8, a deep learning model designed for real time object detection, classification, and segmentation. It detects in real-time using a webcam, the surrounding view of the prosthetic, specifically looking for common handheld objects. Upon detection of said handheld objects, a communication will be sent to that e Arduino via the serial port, that the servo motors should be actuated. The Mobil app was devised using android studio, which combines java or kotlin coding with statistical analysis report using R. Grip strength is measured using a common pressure sensor, where voltage readings are observed across a common voltage divider.

Challenges we ran into

It was difficult to develop a fully cohesive device which consisted of both complex software, and delicate hardware. We kept in mind the primary audience for this device (the prosthetics community) so we took extra care with the materials involved. We had originally planned for a 3D printed device, however after some complications, we had to get creative with what we could obtain, and use the materials available to us.

Accomplishments that we're proud of

Developing the physical device by hand during a short time period is a great accomplishment which we are proud of. Also the cohesiveness and smoothness of the software-hardware connection is something that we are very pleased with.

What we learned

We learned that utilizing our individual strengths to work together leads collaborative success despite certain challenges.

What's next for PRAP - Physical recognition activated prosthesis

To develop a 3D printed model for this device and improve its mechanics. Improve speed and accuracy of abject detection. We would also like to offer this this device (and future models) to amputees for their feedback and perspectives.