Ghostkeys

Inspiration

Look down at a MacBook. The palm rests, the speaker grilles and the strip above the keys make up a big part of the top surface, and none of it does anything. Meanwhile we spend the day reaching for shortcuts we half remember. We wanted to know if that dead space could become buttons, without buying anything.

It turns out Apple silicon MacBooks have a motion sensor inside them that reports 800 times a second and can be read without admin rights. A fingertip tap on the palm rest shows up in it. So does typing, the trackpad, and picking the laptop up. The whole project is about telling those apart.

What it does

Ghostkeys lets you tap the empty parts of your MacBook to run things.

Tap the left palm rest to mute a Zoom call. Double tap the right grille for a screenshot. Tap the top strip to jump to your last app.
Each spot can run a keyboard shortcut, open an app, run a script, or change volume and brightness.
Different apps get different layouts, so the same tap can mean one thing in Excel and another in Figma.
A one minute calibration learns how you tap, including how fast your double taps are and whether you are at a desk or on your lap.
Optional sonar: the speakers play two tones too high to hear, and the mic listens to how your hand changes them, for push, pull and hover in the air above the keys.
Nothing leaves the laptop. No sensor data, audio or taps are sent anywhere.

How we built it

A Swift background service reads the motion sensor through IOKit, finds the start of each tap against a noise floor that adapts to the room, and ignores anything that happens while you are typing, touching the trackpad or moving the laptop.
A tap classifier turns each tap into 33 measurements and decides which spot it came from with two models voting together. If a tap does not look like anything it was trained on, it refuses to guess. It keeps separate models for desk and lap.
Sonar tracks the phase of two inaudible tones, with a hard limit on speaker level so it can never stress the hardware.
An Electron and React app for calibration, a tap tester, a "why didn't that fire?" helper and a live view of what sonar is sensing. It talks to the service over a local connection that needs a secret token and refuses web pages.
The website is Next.js with a real time 3D MacBook in three.js, driven by scroll.
The film was made in Remotion with an ElevenLabs voiceover and a score we generated ourselves.

Challenges we ran into

A tap is tiny. A soft palm rest tap is a few thousandths of a g, about the same as a key press or a mug set down on the desk. Early versions fired on everything. We recorded our own sessions and built a benchmark from them, and every change had to beat it on real recordings, not made up data.
Desk and lap are different worlds. A calibration made at a desk recognised none of our lap taps, because the laptop rocks on a soft lap and people hit harder there. Separate models per position took lap accuracy from 0% to 84%.
Sonar heard nothing at first. After measuring on the actual hardware we found that four of our own safety filters were throwing away every sample. The mic and speakers were fine.
Not hurting the Mac. The service puts every sensor setting back when it exits, caps speaker output, and survived a 30 minute attack test with garbage input.

Accomplishments that we're proud of

On our real recordings, wrong or false taps went from 16.5% to 1.4%.
Picking up and handling the laptop went from about 2 false taps a minute to none.
Over 280 automated tests plus 117 end to end tests.
It works on MacBooks people already own. There is nothing to buy.

What we learned

Numbers from fake data lied to us every time. The only results we trust now are ones measured on real recordings and checked by someone who did not write the code. That habit caught mistakes in our own analysis more than once.

What's next for Ghostkeys

Sonar that works at normal volume, plus left and right swipes in the air
Learning from everyday use, once we can prove it will not learn the wrong thing
A signed public release, and the Windows version that already speaks the same protocol