Assist

Inspiration

Those with limited mobility may have difficulty reaching, picking up, or moving objects around their workspace. I wanted to build an assistive device that could perform these physical tasks without requiring the user to operate complicated controls. One inspiration I got was the large industrial arms used to assemble cars.

What it does

Reach operates across more than 130,000 cubic centimeters of workspace. Its arm-mounted camera identifies and locates objects on a table, while its five degrees of freedom allow it to reach, pick up, hold, sort, and reposition them. Users communicate with it via a Qwen-powered agent.

How we built it

The arm is controlled by a Raspberry Pi 4 running Linux and Python. Its DC-powered joints use TB6612FNG and DRV8876 motor drivers, with potentiometers and magnetic encoders providing position feedback. PID control and inverse kinematics coordinate smooth, accurate multi-joint movement. A PCA9685 controls the servo-powered wrist and gripper, while an ADS7830 handles analog feedback.

For perception, I used OpenCV and an arm-mounted USB camera to identify and localize tabletop objects. The user interface was built with Astro and JavaScript and communicates with the Qwen API. HTTP endpoints connect the interface on the user’s computer to the arm controller on the Raspberry Pi.

The mechanical system—including the five-DOF structure, end effector, wiring, power distribution, and rubber-band counter-spring system—was designed and assembled from scratch.

Challenges we ran into

My greatest challenge was making object detection and localization reliable. Because the camera is mounted directly on the arm, shaking and joint backlash caused the image to move while scanning. I reduced this by tuning the PID controllers for smoother motion and minimizing mechanical backlash.

Glare, changing lighting conditions, and objects placed directly beside one another also made detection difficult. I improved reliability by refining the OpenCV color and contour filters, combining color, shape, and size information, and analyzing multiple frames instead of trusting a single image.

Accomplishments that we're proud of

I'm very proud of how mechanically robust the arm is.

What we learned

I was pushed to become much better at computer vision.

What's next for Assist

Next, we want to improve object detection under a wider range of lighting conditions and support objects the system has not been explicitly programmed to recognize. We also plan to add more manipulation abilities, voice input, improved collision avoidance, and additional safety checks. Ultimately, I want to develop Reach into a more reliable and adaptable assistive device for everyday workspaces.