Rock Climbing Hold Finder

Front_view
Angled _view
Hardware_parts_disassembled
Old_segmented_alg
New_segmented_alg
Holds Labeled on whiteboard, Image from camera
Holds Labeled on Climbing Wall

Inspiration

Climbing became a meaningful hobby for us when we started university. No matter how the day went; classes, exams, or early-morning fire alarms. The climbing wall was always a place to reset. While climbing with our friend, we noticed he struggled to distinguish between red and green holds. He later told us he was colorblind. That realization inspired our project: we wanted to make route reading more accessible so he could enjoy climbing independently.

Project Overview

Our system captures an image of the climbing wall, identifies hold colors through image processing, and uses hardware to point a laser pointer to the holds that are valid to use. This project was designed for colorblind rock climbers.

Hardware

We built a two-axis laser pointer using two mounted servo motors, one controlling horizontal movement and the other vertical. The system is powered and controlled through a breadboard setup with external batteries. A Raspberry Pi 3B serves as the main processor, paired with a Raspberry Pi Camera Module 2 for image capture.

Software how we built:

How we built it

We explored many methods of hold-color detection and image-preprocessing. We built the software to take in an image and return a set of x-y pixel positions that are the centers of climbing holds on the same climb. We first do this by clustering all colors on the image with k-means and using deltaE_ciede2000 to merge similar colors. Then, we explore all connected components of the image to determine the hold volume. We see this visually by replacing each pixel in the input image with its k-means representative. Finally, we do some post-processing to ignore image artifacts and walls. At the end, we get a list of x-y positions that is then converted to the angle measurements the servo motors rotate to.

Challenges we ran into

Our first major challenge was getting the servo motors to work. As beginners with the Raspberry Pi, we initially powered both servos directly from the Pi, which doesn’t supply enough current. The motors wouldn’t move consistently. We resolved this by switching to a breadboard setup with an external power supply and a shared ground.

Additionally, we encountered:

Faulty or inconsistent hardware components
Noise and variation in camera images
Limited access to equipment and space for testing
Running out of resources for our Raspberry Pi

Despite these challenges, each problem helped us refine our approach and deepen our understanding of both hardware and computer vision logic.

Accomplishments that we're proud of

Creating a Cohesive System: We integrated multiple hardware and software subsystems. The success of this integration is evident in our driver file, where all components work in concert seamlessly.

Proprietary Image Processing Pipeline: From inception to implementation, we engineered a custom image processing algorithm that forms the core intelligence of our device.

Tackling Constraints: Limited resources became a catalyst for innovation. We are proud of our ability to improvise and overcome repeated challenges with hardware availability.

What we learned

We gained significant experience in optimizing machine learning models for resource-constrained environments, successfully deploying them to a microcontroller. Furthermore, we developed a comprehensive understanding of the Raspberry Pi ecosystem, from initial setup and configuration to integrating peripheral microcontrollers and executing automated scripts.

What's next for Rock Climbing Hold Finder

Deploy Full Algorithm on the Raspberry: Getting a Raspberry Pi 5 to handle data-processing that the algorithm does. Alternatively, improve algorithm efficiency so the Raspberry Pi 3b can handle the computations.

Enhanced Enclosure & Manufacturing: Designing and producing a more robust and professionally manufactured casing for all components.

Refined Servo Control Algorithm: Improving the servo's accuracy and reliability by implementing a more precise mathematical model and eliminating manual offsets.

Advanced Distance Sensing: Leveraging the integrated ultrasonic sensors with enhanced data processing to achieve more reliable and accurate distance measurements.

Create a more embedded environment: having the parts and components connect to each other nicely without having to plug in and change multiple components to get it to work, It should be one simple button click and everything will run.