Nebius Personal AI

Nebius Personal AI is a personal assistant which is always on and private. You can speak to it; it recognizes how you sound, reasons with an NVIDIA Nemotron open model served on Nebius Token Factory , acts through a set of tools you control, remembers you In different sessions, it responds in a voice that matches the current mood.

It runs as two surfaces from one core: Hosted web app - a public URL where anyone can type or speak (browser voice through the Web Speech API) This is the reliable and judge-testable example: reasoning + No need to install any software, with the use of tools, memory, and web search. Local voice pipeline - the full experience shown in the video: everything except the language model runs "on-device" - speech-to-text (Whisper), speaker identification (ECAPA-TDNN), acoustic emotion detection, barge-in (you can and an optional cloned voice, with the reasoning The prompt leaves the machine and goes to the open model which you can self-host.

What it can do Persistent memory - remembers facts you tell it and recalls them in later turns and later sessions. Notes - note something…, what are my notes?, search your notes. A web search involves looking up information online via the Tavily API when you make a query. Emotion-aware replies - it determines the user's mood and intent then adjusts its tone accordingly. Speaker-aware - it is able to determine who is speaking (on the local pipeline).

Why it fits Personal AI: it's a private assistant with persistent memory, reusable skills (tools), access to the information you choose, and the ability to carry out tasks - with your biometrics and data staying on your own machine while The reasoning is based on an open model.

How we built it (tech + where Nebius/NVIDIA/Tavily are used)

The reasoning brain of the NVIDIA Nemotron on the Nebius Token Factory is the app's LLM client speaks the OpenAI-compatible Chat Completions API, and Token Factory is OpenAI-compatible, so pointing the whole system at Nemotron was configuration, not new code: provider: openai_compatible, base_url: https://api.tokenfactory.nebius.com/v1/, model: The Nano model deals with quick, everyday tasks; the Super model does so as well. Ultra provides a one-line shortcut for more extensive reasoning. One structured call per turn returns mood, intent, sensitivity, the reply, and an optional tool action — so perception, reasoning, and tool-routing happen During one trip of the Nemotron. Tavily powers theweb_search` tool — a real runtime call to the Tavily search API returning a synthesized answer plus sources for the assistant to speak. On-device perception (local pipeline): faster-whisper (STT), SpeechBrain ECAPA-TDNN (speaker ID), an acoustic emotion classifier, Silero VAD, and The system uses Coqui TTS and its memory is based on SQLite. Hosted surface is deliberately torch-free (browser does the voice), so it You can get them free and the system will wake up in seconds.

What was significantly updated during the Submission Period (disclosure)

This project composes two of my own pre-existing projects - voice-agent (a real-time voice pipeline) and offline_assistant (a task-automation tool The suite also predates the hackathon. Everything which makes this a submission was built during the Submission Period, in a fresh repository (the originals are untouched):

The merge itself - a new adapter that bridges offline_assistant's tool suite (notes, files, system info, activity report, code check) into voice-agent's provider-agnostic tool registry, so one assistant has both the The voice front-end and the task tools. The entire Nebius + Nemotron integration - pointing the OpenAI-compatible client at Token Factory, model selection, remote-key handling, and a critical latency fix (detailed thinking off + a token cap) that cut Nemotron voice Decreases from about 51 seconds to about 6 seconds.

The Tavily web_search tool — a new integration within the runtime. The lightweight hosted web app - a new FastAPI surface with browser-based voice (Web Speech API), a curated public-safe tool set, per-session memory, and A deployment on Docker and Hugging Face Spaces. Reliability + correctness work - a system-voice TTS backend (so the app doesn't depend on the hard-to-install voice-cloning stack), a SQLite notes store, a push-to-talk capture fix, dependency pinning (numpy<2 for torch), Taking the numba/llvmlite/librosa dependency chain out of the core and the tests.