Backend Pipeline
Circuit Graphics
Circuit Graphics
Circuit Graphics
Website
Website
Website
Website

Buddy — Project Story

Inspiration

We've all been there.

It's 9 PM. You have a deadline at 10. You open your laptop with full intent and somehow, forty-five minutes later, you're three rabbit holes deep into a YouTube spiral you didn't choose to start.

Distraction isn't a character flaw. It's a design flaw. The apps engineered to steal your attention are built by thousand-person teams with billion-dollar budgets. You're just one person trying to finish your important task for the day.

We dreamed big and kept cutting short; not from laziness, but from the trap of instant dopamine. Every notification, every autoplay, every algorithmically perfect next video is optimized to beat the thing you actually care about.

So we asked: what if you had something on your side?

Not another screen time report. Not another app telling you what you already know. A guardian angel. A mentor who actually knows what you're trying to accomplish today; because you told them; and who checks in when you drift, not with a silent badge, but with a voice.

That's Buddy. An always-on, physically present AI companion strapped to your wrist. You speak your intentions to it. It holds them. And when the gap between what you meant to do and what you're actually doing gets wide enough to matter, it speaks back.

Not to shame you. To remind you of the version of yourself that set that goal an hour ago.

What it does

Hold the button. Say your goals out loud:
"Hey Buddy — email my professor, then senior design until 10."

Buddy stores them. Then it watches. Every app you open on your laptop is scored in real time against what you said you'd do. The moment you've been in a game or a scroll for long enough to cost you, the speaker on your wrist fires:

"You have senior design until 10. You can play after that.
You'll regret this block later — go back to the project."

No banner. No vibration you'll ignore. A voice. Your goals, reflected back at you.

Every session becomes a scored timeline, scored my Local AI model trained in house — stored in a time-series database, plotted live in MATLAB.

How we built it

Five systems built in parallel over 36 hours, each owned by one person, all meeting at a shared API contract we froze on night one.

The watch — a LilyGO T-Display-S3 (ESP32-S3) with an I2S MEMS microphone, MAX98357A amplifier, and a 3W speaker. Press the button, speak, release. Raw PCM streams over Wi-Fi to the laptop backend; the reply streams back and plays through the speaker with no buffering delay.

The voice layer — ElevenLabs Scribe v2 for speech-to-text, ElevenLabs TTS for the reply. All server-side. No API keys on the watch.

The brain — Gemini with persistent task memory. Every voice call carries the full task list so Gemini always knows your goals. A separate coaching endpoint generates the intervention line when the PC agent decides it's time.

The PC agent — scrapes the foreground window every second via Windows APIs, runs a local productivity classifier (0–1 score), feeds a TimescaleDB hypertable via Tiger Data, and writes a CSV that MATLAB plots live. The trigger logic lives here — rolling average, streak timer, cooldown — so Gemini only ever writes sentences.

The dashboard — a Lovable web app showing the score timeline, task list, incident feed in real time.

Challenges

Two I2S buses, one chip. Shared clocks between the mic and amp flood the audio with noise. Running I2S0 and I2S1 on fully separate GPIO pins with no shared signals killed it — but only after two hours of unexplained silence.

Streaming audio on an ESP32. When FastAPI sends chunked transfer encoding, HTTPClient::getSize() returns -1. malloc(-1) crashes the board. We replaced the buffer-then-play approach with a direct stream-to-I2S pipe that never allocates for the full response.

Making Gemini brief. A 3W wrist speaker does not want a paragraph. Hard constraints in the system prompt — one line, name the deadline, no moralizing — turned reflective Gemini into a sharp, human-sounding coach.

What we learned

The gap between intention and attention is measurable. Once it's measurable, it's closeable.

And a voice hits differently than a notification. You can swipe away a banner. It's much harder to unhear your own goals read back to you.

What's next

Wake-word detection — so the button disappears entirely.
Android/ IOS app.
Reduced device size close to whoop band. Learned per-app scoring — so the model knows you, not just a generic rubric.
A weekly arc — not just what you did each day, but whether you're becoming the person who does what they meant to. AI agents that can do your tasks.

Buddy isn't trying to run your life. You tell it the day. It remembers, it watches, and it talks you back to the work.