NORA

An autonomous assistant that sees, hears, and directly controls your machine to get things done.

Comment

🎙️ Nora: Breaking the Glass Ceiling of AI

A journey from text-boxes to true autonomous agency.

💡 Inspiration: The Prison of the Predefined

For years, software automation was a tax on human creativity. If you wanted to automate a task, you had to predict every outcome, write every line of logic, and anticipate every failure. It was exhausting and rigid.

When LLMs emerged, we saw a rift in the universe. Suddenly, we had "brains" that could reason—but they were trapped. Locked inside a Text Box Paradigm, a digital cage where they could talk but never touch; advise but never act.

Nora was born from a singular, dramatic question:

"What happens when you give the ghost a body?"

We built Nora to be more than a tool. She is a companion—a live, autonomous agent that breaks the keyboard barrier and interacts through the human senses: Voice, Vision, and Action.

🌟 What Nora Does: The Ghost in the Shell

Nora is a full-scale autonomous presence living on your machine. She is a voice-first assistant that doesn't just suggest a solution—she becomes the solution.

  • 👂 She Hears & Speaks: Real-time bidirectional audio with sub-second latency. No "Push-to-Talk" needed—just speak naturally.
  • 👁️ She Sees: Share your screen, and Nora scans for error codes, messy desktops, or broken UI elements. She understands your context visually before you finish your sentence.
  • 🛠️ She Acts: Nora commands a persistent, stateful "Live Command Engine." She can reach into your OS, scaffold projects, refactor code, or organize a chaotic desktop into categorized folders while you watch.
  • 🌎 She Adapts: Whether on macOS or Windows, Nora senses her environment and arms herself with a tailored suite of tools to fix your world.

🏗️ How We Built It: Orchestrating the Machine

Building Nora was like conducting a digital orchestra, blending cloud intelligence with low-level system execution.

The Architecture Workflow

Nora Architecture Diagram

The Technical Backbone

  • 🧠 The Brain (Google ADK & Gemini): We utilized the Google Agentic Development Kit (ADK) to manage complex conversation states while leveraging the native audio capabilities of *Gemini 2.5 Flash Native Audio *. The Bidi-Streaming API is the heartbeat that makes her feel "Live."
  • ⚡ The Nervous System (FastAPI): A high-performance FastAPI orchestrator acts as the high-speed bridge, synchronizing audio playback and managing the "Tool Interceptor" queue for real-time local execution.
  • 🦾 The Hands (Secure Client Daemon): We engineered a custom Python Daemon that spawns real, persistent terminal sessions. This allows Nora to navigate your file system with the continuity and memory of a human technician.
  • 🎬 The Cinematic UI (React 19): Built for immersion using AudioWorklets for lag-free PCM processing and Framer Motion for the "hacker-smooth" Live Activity dashboards.

🛡️ Challenges: The Great Linking Battle

The path to autonomy wasn't easy. Our greatest dragon was the "Linking Problem." Securely controlling a local machine from the cloud reliably is a massive hurdle.

We battled WebSocket drops and state resets until we engineered a custom Persistence System and a robust Tool Interceptor. When Nora moved a file on our command for the first time, it didn't feel like a successful build—it felt like magic.

🏆 Accomplishments & Pride

We have shattered the "Chatbot" stereotype. Hearing Nora’s voice as she autonomously navigates a complex folder structure or seeing her compile a Python app she wrote herself is more than a technical feat—it’s a glimpse into the future.

🎓 Lessons Learned: Beyond the Box

We learned that the "Live" aspect of an agent—the ability to interrupt, to see, and to act instantly—is the difference between a search engine and a partner. Nora is free of the text box, and we are never going back.

🚀 What's Next for Nora

Nora is only just waking up. We are advancing her GUI interaction capabilities and expanding her system toolbelt until the boundary between your voice and your computer’s actions completely disappears.

Built with ❤️

Built With

Share this project:

Updates