Inspiration Sometimes you need to communicate when speaking would be difficult or draw attention. We imagined a team coordinating quietly across a room and a person trying to get a message across on a loud street. In both cases, what if the watch or strap already on your wrist could be the input? That became AirType. We wanted someone to enter a short message with wrist movements and have it appear on a screen or play through a teammate’s earbuds. The longer-term idea is to use the same input to talk to an AI agent without speaking.
What it does AirType turns a WHOOP 5.0 into a silent keyboard. It recognizes six movements: roll in, roll out, tilt up, tilt down, swing left, and swing right. Each movement represents a group of letters. The macOS app has Start, Finish, and Clear controls. You can enter a rehearsed shortcut or gesture the first three letters of each word, using the whole word when it’s shorter. AirType can show the result, type it into a focused document, speak it through Deepgram, or type and speak it together. Audio plays through the Mac’s selected output, including connected AirPods.
How we built it 1. Motion capture. We built a Swift Bluetooth capture tool using community protocol documentation. It verifies frame checksums and records three accelerometer axes and three gyroscope axes at about 100 samples per second, preserving raw values, sample counts, and timestamps. NOOP protocol notes (https://github.com/ryanbr/noop/blob/2a1de7609843b095505274ad6fc811757210be80/docs/PROTOCOL_SENSORS.md#r21-six-axis-imu) 2 . Gesture recognition. We recorded labeled gestures, mixed sequences, and ordinary movements such as walking, drinking, and repositioning the arm. Our logistic regression model uses features from short motion segments, including rotation direction and speed, acceleration statistics, and duration. 3 .Sentence decoding. AirType checks for a rehearsed shortcut first. For new sentences, a local dictionary proposes words and where to put spaces. OpenAI receives those candidates and the full gesture sequence. AirType checks that the suggested sentence encodes back to the observed movements within one gesture edit before delivering it. 4 . Output. The floating panel shows the gesture sequence and decoded result. The app can type into the focused window and send finished text to Deepgram for playback through the Mac’s audio output. Six gestures give us only six distinct signals per move. A fixed code with a unique combination for every letter would take two gestures per letter, since six choices cover only six letters and two gestures give 36 combinations. Grouping letters lets us use one gesture per entered letter and resolve ambiguity with word and sentence context.
Challenges we ran into • Bluetooth pairing. Connecting the strap to the Mac can disrupt its phone pairing. We made capture opt-in and tested restoring the phone connection. • Batched motion data. The strap delivers roughly one second of samples at a time. We preserved the original timing and sample counts and skipped rows without usable sample times. • Repeated gestures. A second tilt-up can get lost, and the return movement can register as tilt-down. That matters when two letters use the same gesture. • Ambiguous input. Different words can share a gesture sequence, and an extra gesture can change the result. We use the dictionary for candidates, OpenAI for sentence context, and a consistency check before delivering the text. Accomplishments that we’re proud of • Captured live six-axis WHOOP motion on a Mac at about 100 Hz, with zero framing rejects in our successful validation runs. • Recognized 43 of 44 reviewed gestures and 14 of 16 exact sequences on a separate mixed recording. The model also produced three false commands during reviewed background motion. • Ran all three rehearsed shortcuts through the live macOS panel and logged accepted OpenAI suggestions from live wrist input.
What we learned
A classifier can do well on individual movements and still struggle when they’re chained into a sentence. Once we started testing sequences, repeated gestures, ordinary return motions, and Bluetooth delivery delays mattered as much as the label on any single movement. Saving raw motion and logging decoder results let us tell where each failure happened.
- What’s next for AirType We want to make repeated gestures more reliable, test with more wearers and strap positions, and measure how often new sentences come through correctly from wrist to text and audio. The model was trained on one wearer’s left wrist. Our longer-term goal is to let someone gesture a request to an AI agent and hear its reply through an earbud. Today, AirType produces the text of the request. Executing it and delivering the agent’s reply are future work.
