Inspiration
Every time a text comes in, we pick up our phones, and five minutes later we're scrolling instead of working. We wanted to stay in the loop without touching the phone at all. We started the weekend building a desk whose wireless charger follows your phone around, and realized the part we liked most was the little assistant on the desk edge. So we turned it into DeskBuddy.
What it does
DeskBuddy is a desk companion built on a FREE-WiLi that reads your iMessages to you, only when you ask.
- A message arrives → the LEDs light up blue, it chimes, and the screen shows you've received a message
- "Read my message" → Reads the newest message out loud
- Clicking the green button + "Tell me my message" → Reads the newest message out loud
- Clicking the green button + "Repeat" → Repeats your most recent message
- Clicking the green button + "Summarize my most recent messages" → Gives you a short AI-generated summary of your most recent messages instead of the whole thread.
- "Missed Call" → Alerts you after the call has ended and indicates who you received the call from.
- On-screen GUI → A clean, sleek interface on the FREE-WiLi's own display shows your incoming messages, summaries and DeskBuddy's status at a glance.
Your phone stays face down, and nothing is read aloud until you ask.
How we built it
- FREE-WiLi is the hardware: buttons for push-to-talk, RGB LEDs for status, the screen for DeskBuddy's GUI, and its speaker for the voice. A Python program on the laptop drives it over USB with the
freewililibrary. - ElevenLabs is the voice behind DeskBuddy. We convert the reply to a WAV file, upload it to the FREE-WiLi, and play it from its own speaker.
- Messages: The FREE-WiLi's Bottlenose Orca pairs with your iPhone over Bluetooth Low Energy and subscribes to its notifications, the same way a smartwatch does. When an iMessage notification arrives, the Bottlenose grabs the sender and text and hands them to DeskBuddy. Nothing to install on the phone.
- Summaries: Summaries are generated by Google Gemini, which condenses your most recent messages into a sentence or two that DeskBuddy reads aloud.
- GUI: We designed a clean, sleek interface for the FREE-WiLi's 320×240 screen that shows what DeskBuddy is doing in real time: new messages, summaries, missed calls and its current status.
- A small event bus ties it together, so button presses, voice commands and new messages all go through one "brain" that decides what to do.
Challenges we ran into
- Pivoting late. We spent most of the weekend on computer vision and a stepper-motor gantry but ultimately had to step away from that idea due to time conflicts.
- Audio on the FREE-WiLi: playing the ElevenLabs voice from the device instead of the laptop meant converting and uploading audio fast enough to feel responsive.
Accomplishments that we're proud of
- A fully voice-controlled assistant running on a device the size of a deck of cards
- Getting a natural ElevenLabs voice to come out of the FREE-WiLi's own speaker
- A clean, sleek GUI running directly on the FREE-WiLi's screen, so DeskBuddy needs no extra display
- A privacy-first design: messages are only read when you ask
- Building working computer vision and motor control along the way, even though we pivoted
What we learned
- How to talk to embedded hardware over serial, from simple step commands to the FREE-WiLi's event API
- How to chain speech-to-text, AI and text-to-speech into a loop that feels quick
- Practical computer vision: homographies, lighting and false positives
- That a focused, polished feature beats a big half-working one, and when to pivot
What's next for DeskBuddy
- Replying by voice: "Tell Alex I'm on my way"
- Running without a laptop, using the Bottlenose Wi-Fi extension so DeskBuddy works anywhere on the desk
- Focus mode: hold non-urgent messages during a study session and summarize them at the break
- Wearable mode on the FREE-WiLi's watch strap
- More sources: calendar reminders, email and notifications
