WONDERWALL·E

The WonderwallE
The WonderwallE (side view)

Inspiration

Produce a musical cobot that syncs with you so you don’t have to stop your flow.

What it does

WonderWall·E is a cobot that teaches you to play guitar in rhythm to the music and maintain a consistent beat. It can serve as a visual metronome that dances along to the music, learn your strumming pattern and play a learned strumming pattern on loop.

How we built it & challenges we ran into

SO-ARM100 dances to music A song was downloaded in MP3 format. The librosa Python library was used to extract beat timestamps in seconds from the song. The python-vlc Python library was installed and used to play the song and respond to beats live. Experimental offsets were determined to make the song playback synchronize with the beat reports. Beat timings were then used to drive robot movement through phosphobot's HTTP API. Recordings of puppeteered movement were made to create more sophisticated dancing.

Arm controls & usability improvements Due to several servomotor failures, alternative arm control methods were developed in order to control arm joints without using the the Phosphorobot GUI or circumventing the Phosphorobot entirely.

Record & Replay A record & looped replay system was created by capturing joint angles in real time and reproducing them with precise timing. The Phosphobot API was utilized to read joint positions at a configurable frequency. During recording, torque is disabled to allow manual manipulation of the robot arm. Motion is reconstructed by streaming saved angles back to the Phosphobot API with reactive timing to improve accuracy of movements.
Real-time servo motor control A Python script was developed to control any given Feetech STS3215 servomotor using the motor ID. The script initilaizes a motor bus configuration, establishes a connection, and reads the motor’s current position. It then enters an interactive loop, prompting the user to input step values to incrementally adjust the motor’s position.
Joint reduction To minimize the size of the SO-ARM100, the arm was reconfigured to have two appendages so that six motors would be recognised by Phosphobot. The primary end effector strums the guitar and the secondary end effector repeats the same rhythm and act as a dancing sidekick.

Accomplishments that we're proud of

Making music, making friends

What we learned

LeRobot & Phosphobot Intergenerational language & culture exchange (16 yrs -> 32 yrs) Assembly & disassembly skills Robotic hardware troubleshooting

What's next for WONDERWALL·E

Improve strumming accuracy Improve individual motor control More modular More lightweight Additional fretting arm Looper pedal interface (record & replay hotkeys) Live dancing