Inspiration
Learning guitar assumes two hands: one frets, one picks. People with a limb difference, an amputation, a stroke or a nerve injury often have one working hand, and the instrument quietly shuts them out. Adaptive guitars exist, but they're usually custom-made, expensive, or limited to a pick strapped to the arm, which can only strum.
DurHack's theme was mixing art, music and tech. We wanted to combine two things that don't usually go together, a robot hand and a human hand, on one instrument. The player keeps the expressive half, choosing every chord; the robot does the half they can't.
What it does
Servo Strummer clips onto an ordinary nylon-string guitar. Six servos, one per string, sit between the soundhole and the bridge, each swinging a short length of wire across its string to pluck it.
- Strums and fingerpicks. 14 patterns, from a plain strum to Travis picking, arpeggios, boom-chick, reggae skank and a "Super-human" 32nd-note cross-picking pattern no single hand could play. There's also a Random mode that invents a new bar every bar.
- Follows the chord you hold. It knows 20 chord shapes. When you set a chord, it skips the strings that chord mutes (no low E on a D chord) and the bass notes follow the root.
- Listens to the guitar. "Chord by ear" uses the laptop microphone to work out which chord is ringing from the notes in the spectrum, then tells the robot. Fret a new chord and it follows within about a bar, with no training.
- Takes voice commands (ElevenLabs). Your only hand is on the fretboard, so voice is the free input. Hold a button (or the space bar) and say "play Travis picking", "faster", "tempo ninety", "chord A minor", "how do I play F", "twelve bar blues" or "chords G, D, E minor, C". It answers out loud, and while a song plays it calls out the next chord a beat before the change. It also works as a coach: ask it about a song or a fingerstyle pattern and it talks you through it, which chords to hold and when to change. If the room's too loud, you can type the same commands.
- Hardware controls. A knob sets the tempo, a press switches pattern, and an LED matrix on the board shows the current chord.
- Virtual guitar. A live digital twin on the web page mirrors every pluck, string by string, and can synthesise the sound so you can try patterns without the robot.
- Setup tools. Per-string pick calibration, a string test, a servo reset and a debug mode that runs everything except the servos.
How we built it
- Hardware: six SG90 servos on a 3D-printed PLA mount (modelled in code with CadQuery, printed on a Bambu A1), on a classical guitar, driven by an Arduino UNO Q. We didn't have an Arduino power supply, so a Raspberry Pi 4 powers the board instead. The servos pluck with short pieces of wire.
- Microcontroller (C++): the UNO Q's STM32 runs a bar sequencer that plays each bar on 20 ms servo frames with exact integer timing. It ripples strums 20 ms per string, swaps the next bar in at the bar line and applies tempo changes on the next beat. We generate servo pulses ourselves with interrupts off for only around 25 µs at a time, because the standard Servo library's timer jitter made the servos twitch.
- Linux side (Python): the UNO Q's Linux side runs the music logic: patterns, chord voicing, songs, tempo and the knob. It queues bars one ahead over the Arduino Bridge and pushes live state to the page over a socket. Settings and calibration live in SQLite.
- Web page (HTML/JS, no build step): served from the board itself. Chord-by-ear uses the Web Audio API: an FFT, a 12-note chroma from spectral peaks, then matching against each chord's expected notes. The virtual guitar uses Karplus-Strong synthesis.
- Voice (ElevenLabs): speech-to-text turns speech into text, which a local parser maps to commands like play, tempo, chord and song. Replies and chord call-outs come from ElevenLabs text-to-speech and are cached, so repeated phrases are free and instant. The board has no internet, so the page calls ElevenLabs directly, and the API key lives in a secrets file on the board, never in the repo.
- Testing: Python unit tests, JS tests that synthesise strummed chords to check recognition, voice-command tests, and C++ sequencer tests that check the same golden vectors as Python. All of them run in GitHub Actions on every PR.
- Teamwork: three of us each worked with our own Claude Code session, coordinating through GitHub issues and PRs as a message bus.
Challenges we ran into
- Servo jitter. The Servo library's software timer gets delayed by other interrupts, which stretched pulses and made picks twitch mid-song. We replaced it with our own pulse generation and later moved the whole sequencer onto the microcontroller.
- Fitting six servos across 11 mm string spacing. An SG90 is wider than the gap between strings, so the servos sit in two staggered rows of three. Our first print didn't fit the guitar, so we iterated on the mount.
- Timing. A robot strum sounds mechanical if every string fires at once, so strums ripple string by string. The servos also need about 160 ms to swing back, which the fast patterns have to respect.
- Weak servos. SG90s don't have much torque, and a normal pick snagged on the strings instead of plucking them. We started with bits of card, moved to toothpicks, and finally settled on short pieces of wire, which are stiff enough to sound the string but flexible enough for the servo to push past it.
- No power supply. We didn't have access to a power supply for the Arduino, so we powered it from a Raspberry Pi 4 instead.
- A loud hackathon hall. Chord recognition and voice both rely on microphones beside a strummed guitar in a crowded room. We kept the mic streams separate (voice processing on for speech, off for notes) and added typed commands as a fallback.
Accomplishments that we're proud of
- It plays real fingerstyle patterns, not just strums, which is what a strapped-on pick can't do.
- Fret a chord and the robot follows by ear.
- Timing runs on the microcontroller and is tested on both the Python and C++ sides against the same vectors.
What we learned
- Real-time control on a microcontroller: interrupts, pulse timing and double-buffered bars.
- Music signal processing: chroma features and why harmonics matter for recognising chords.
- Designing input for someone whose only hand is busy: voice, a knob, and following what they play instead of asking them to press buttons.
What's next for Servo Strummer
- Test it with one-handed guitarists and adaptive-music organisations, and let their needs drive the design.
- More hands-free triggers: a foot switch, or a motion controller strapped to a wrist, foot or residual limb.
- Proxy ElevenLabs through the board so the key never leaves it.
- A sturdier, guitar-agnostic mount that clamps on without tape.
