BioSync

Github commits
Pic with Neil Armstrong!
MyoWare EMG
Initial ESP32 & IMU testing
Wireless LeRobot Arm Test
Wireless LeRobot Arm Test 2
Goofing around
Attaching devices for LeRobot arm

Inspiration

The initial inspiration for BioSync was to explore technologies that assist bedridden and severely ill patients lacking basic motor functions such as limb movement. However, we have also expanded into other fields, such as, home-comfort and precision engineering.

What it does?

Our project uses a human-computer interface with the help of sensors, microcontrollers, and hardware to help wirelessly automate certain human and living processes. The SO-100 is completely wirelessly operated using a human arm and the Smartpulse ring light automatically adjusts brightness based on user rest/active pulse.

How we built it?

We began by studying the AMD SO-100 GitHub repository, running test scripts on the main controller to understand the arm's full capabilities. We narrowed focus to core tactile movements like arm rotation, extension, gripper open/close, and wrist movement, and used the provided Python leader-follower script as our foundation. After further brainstorming, we realized we were centering our thoughts around the arm's capabilities too much rather than our original bio-assistance motivation, so we pivoted to incorporate heartbeat monitoring and sleep cycle lighting from our available components. For wireless control, we started with Wi-Fi but quickly ran into issues with the 2.4 GHz limit and network traffic, so we switched to BLE. We picked an ESP32 and BM008x IMU as our initial microcontrollers, but the BM008x had programming issues, and the MPU6050 we tried next was missing magnetometer and gyroscope features. With no other IMU options at the time, we shifted to basic directional movements on a breadboard. By luck we acquired 3 MPU9250s, using one for the upper arm and one for the forearm, and achieved 5 degrees of freedom wirelessly. Over the following hours we refined shoulder planning, improved coordination between IMUs, reduced latency, and increased movement speed and range of motion. Our final improvement was achieving a decent degree of wrist torsion.

For the SmartPulse ring light, we ripped apart a standard ring light from the hardware storage and installed an ESP32 as well as other electronic components to automate it. The smart light is controlled using data from a heartbeat sensor. Resting pulse translates to a dimmer light and active pulse translate to bright light.

Challenges we ran into?

We ran into a number of hardware and scope challenges throughout the hackathon. Wi-Fi proved unreliable due to 2.4 GHz congestion and network traffic, forcing a switch to BLE. On the IMU side, the BM008x had programming compatibility problems and the MPU6050 was missing magnetometer and gyroscope support, leaving us without a working IMU for a stretch before we were able to source MPU9250s. We also initially assumed a single IMU could control all 6 degrees of freedom on the SO-100, but found we needed 2 to 3 to cover the full range of motion. The Myoware Muscular Sense 2.0 worked well in isolated testing, but we were unable to integrate EMG as the 6th degree of freedom in the final implementation. Early on we also had to course correct after realizing we were building around the arm's capabilities rather than our original bio-assistance mission.

Accomplishments we're proud of?

We were extremely proud to see our brain child come to life. It showed that all the theory and technical skills we've learned throughout our college courses were able to put together an innovative product.

What we learned?

This hackathon was a first in a lifetime experience. Nearly 36 hours of continued problem solving is extremely draining but equally rewarding. We were able to quickly identify our strengths and weaknesses and work on them during the duration of this experience. We were able to brush up on and improve several technical skills. Additionally, we learned how to frame a problem statement, identify material and monetary needs, scale down ideas that extend beyond the scope of our resources and work efficiently while facing a tight deadline.

What's next for BioSync?

We hope to proceed with future projects iteratively using learnings from this hackathon as well as better components to expand the scope of what is possible with human-computer interfacing and automation.