-
-
Home Screen
-
Chat window
Inspiration
Most voice assistants trade convenience for dependence on distant servers. We wanted something different: a private assistant shaped around one real home, running on a single 16 GB Apple Silicon Mac, with clear boundaries between local processing and deliberate online research.
Hoshi is not intended to be a universal commercial assistant. It is a personal system that understands its own home, exposes how it reached an answer, and admits when it does not know.
What it does
Hoshi provides spoken interaction through a browser and a Home Assistant Voice satellite. Wake-word detection, Whisper speech recognition, a Gemma-based language model, memory, and selectable speech synthesis can run locally.
It can answer questions, control the home, manage timers, remember household context, and research online when explicitly requested or approved. Online answers include sources instead of presenting old model knowledge as current information.
A default-deny capability kernel protects actions that change external state. Sensitive systems—including memory, cloud access, and speaker recognition—are flag-gated.
How we built it
Hoshi uses a hexagonal ports-and-adapters architecture. Models and infrastructure live behind replaceable interfaces, while lightweight Python sidecars provide local inference for speech recognition, speaker embeddings, language generation, knowledge retrieval, and speech synthesis.
During Build Week, Codex with GPT-5.6 acted as both an implementation agent and an adversarial reviewer. It contributed speaker-evaluation and safety tooling, a local Piper speech sidecar, repository migration work, and frontend internationalization. A separate orchestrator reviewed, integrated, and independently tested those changes.
The agents coordinated through CollabOS: a small, inspectable, file-based collaboration protocol. They exchange versioned letters through inbox and outbox directories. Requirements, corrections, evidence, and mistakes remain readable instead of disappearing inside a central orchestration service.
Hoshi also gained an opt-in GPT-5.6 research path using OpenAI web search. It is source-aware, model-labeled, configurable, and protected by a daily spending limit.
Challenges we ran into
The hardest constraint was not building more features—it was knowing when not to enable one.
Speaker recognition once associated a browser recording with the wrong enrolled profile. We disabled recognition, froze the available evidence, and evaluated alternative scoring strategies offline. None passed both our recognition and safety gates on the available data, so the feature remains disabled. Uncertainty becomes “guest,” never a confidently guessed identity.
Running the complete stack on 16 GB also required strict resource discipline. Only one brain model may remain resident, sidecars must stay lightweight, and failures must remain honest instead of silently falling back to the cloud.
Accomplishments that we're proud of
We built a complete local voice path that works through a real satellite in a real home—not only through a developer console.
We added explicit, source-backed online research without turning the assistant into an invisible API wrapper. We created reproducible reviewer commands, runtime probes, and an evaluation path that can produce an honest no-go result.
We are also proud of CollabOS. Two AI agents could implement, review, disagree, correct each other, and leave the entire reasoning trail inspectable through ordinary files and version history.
What we learned
A green test does not prove that a living system works. Automated tests must be paired with runtime evidence, realistic input, and clearly stated limits.
We also learned that an AI agent becomes more valuable when it can challenge the premise of a task. Codex’s strongest contributions were not merely generated code, but findings that changed what we measured, what we claimed, and what we refused to deploy.
Finally, local-first is not a single toggle. Every stage—speech recognition, language model, memory, speech synthesis, and fallback behavior—must tell the same privacy story.
What's next for Hoshi
Next, we will collect consented speaker samples and build a proper calibration benchmark before reconsidering speaker recognition. We also want to improve multilingual interaction, evaluate memory retrieval scientifically, and make the first-run local experience easier without introducing silent cloud fallbacks.
Hoshi 0.8 began before Build Week. This submission focuses on the dated, verifiable extensions built during the competition window. The public repository includes setup instructions and a short reviewer-verification path so the central claims can be checked independently.
Log in or sign up for Devpost to join the conversation.