Crux AI

Inspiration: Stemming from personal experience, we are beginner climbers (with the exception of one!) and wanted to innovate a way to expedite the learning process in a fun, autonomous way. When you start out climbing, the learning curve can be pretty steep, there’s a lot of different skills you should learn and incorporate, twisting your body in ways you didn’t know were possible… For some, it may be a daunting and intimidating process. But there’s no better way to learn than direct experience! Even better if you could learn in real time? your own habits and movements, so you can calculate your own personalized method for improvement. After learning a LOT about rock climbing, we developed our project around the dynamics of rock climbing to be as accurate, respectful, and helpful to the climbing process.

How we built it: OpenCap Gemini AI Raspberry PI

How it works: There are two types of inputs: video input from the app and data input from the wearable technology.

The wearable component, which consists of a headless Raspberry Pi, collects physical data such as humidity at the user’s temple (to indicate possible stress levels) and wrist velocity using a single accelerometer. We aim to expand this to include additional extremities relevant to rock climbing.

The raw data is then transmitted via Wi-Fi to the host device, along with the video data. The system processes this information into a chart that is readable within the app, while also providing additional input for the AI to analyze and deliver real-time rock climbing advice.

Challenges: At first, we didn’t know what to use to help guide the user. We contemplated a laser attached to a servo to point at holds, later scratching the idea as we realized most inefficient climbing comes from bad feet, not hold choice (rock climbing lesson 1!), and safety concerns. It was hard to decide what mix of hardware and software to use. We needed a device that could connect to the wearable tech without being invasive to the user, as well as capable of exchanging data with our software.

Uses: Disability accessibility - can aid visually impaired climbers Beginner teacher - guides beginners to start developing the right habits and tricks early on

So far, the current hardware prototype is capable of tracking a single hand, along with a humidity and temperature sensor placed at the temple to monitor potential physical strain. With further development, we would expand the system to include an EEG (adapted from a Mindflex headset and an ESP32 for better readings of stress levels) and incorporate additional accelerometers to map other key areas of the body relevant to climbing.