Inspiration

We'd never built a consumer product before, and we wanted to make something we would actually use ourselves. Our laptops hold everything: our files, our apps, our logins. But the moment we walk away, all of that is out of reach. So we built Rubayeth, an assistant you can call, text, or talk to, so you can get work done from anywhere without touching your computer.

What it does

Call it. FaceTime your Mac from anywhere. It picks up (only for people you trust), shares its screen so you can watch, and gets to work.

"Hey Rubayeth, send my presentation to my email." It finds the deck, confirms out loud, exports a PDF and emails it before you hang up.
"What's on my screen right now?" It describes your desktop while you watch it live.

Text it. No time to talk? iMessage your Mac.

"What's on my screen?" You get a plain-English answer plus a screenshot as proof.
"Find me burgers on Uber Eats." It searches in your already-logged-in browser and texts back the top options, no API needed.
"What was I working on last night?" It recalls your recent activity from its memory.

Talk to it. At your desk, just say what you want.

"Open my sales spreadsheet, find my top five best sellers and make a graph." It opens Google Sheets, loads 400+ transactions, ranks them with a query and charts them. "Now exclude refunds." The table and chart update live.
Speak Bangla and it answers in Bangla.

It remembers your work. It learns your routines and corrections, so the more you use it, the less you have to say.

Designed for people.

Make it yours: pick the language Rubayeth speaks (15 languages or auto-detect), choose from 21 voices with British, American and Australian accents, pick fast or expressive speech, and tune stability, style and speed, with a live preview.
Stays out of your way: a floating panel you can drag anywhere or glide around with Control + Command + arrow keys, and hide or show with Command + Shift + V.
Always shows what it's doing: a status light shows Listening, Thinking or Speaking, and it narrates long tasks out loud.
Safe by default: it only answers calls from people you trust, uses a wake word on calls so background noise never triggers it, and confirms out loud before sending anything.

How we built it

A LangGraph agent server in Node.js routes each request to specialist agents (files, desktop, browser, shopping, notes). For the hackathon the reasoning model runs locally on the Mac; in production it moves to a hosted model.
ElevenLabs for listening and speaking, with the Mac's own voice and Whisper as offline fallbacks.
Gemini reads the screen, finds what to click, translates, and creates embeddings for memory.
Tiger Data (TimescaleDB, pgvector, DiskANN) stores every action, turns them into habits with continuous aggregates, and recalls context by meaning and time.
macOS automation (AppleScript, the Accessibility API, Apple Vision OCR) gives it hands, and BlackHole virtual audio lets it be the Mac's mic and speaker on a FaceTime call.

Challenges we ran into

There's no FaceTime API. We detect the incoming-call banner by its pixels, read the caller from the Accessibility tree, and route audio before answering. Callers heard total silence until we found FaceTime rejects 16-channel audio devices.
Driving Google Sheets with keystrokes was fragile: popups ate key presses. We made it check its own work with on-device text recognition and retry.
iCloud-synced folders stalled file reads for up to 80 seconds until we moved the search off the main thread.

Accomplishments that we're proud of

Our first consumer product, and something we actually use.
A full FaceTime-to-email workflow that works live: someone calls the Mac, asks for a file, and it lands in their inbox before they hang up.
A real spreadsheet analysis done by voice, where the numbers come from formulas, not the AI.

What we learned

Building for real people is a different kind of hard. Speed, trust, and recovering from mistakes matter as much as the AI.

What's next for Rubayeth

Windows and Linux support, so Rubayeth works on any computer, not just Macs.
Text it for files: "Send me my resume" and the PDF lands right in your messages.
Chain workflows from one call: "Build my sales report and email it to me."
Check on long jobs: "Is my export done?" and it reads the progress bar and texts you back.
Tidy up and reply: "Clean up my desktop" or "Reply to Dean's last email saying I'll be there at 5."