Halo

Inspiration

Desktop assistants always felt buried, another browser tab, another app window competing for space. We wanted something that lived on the desktop itself and was always one click away, that felt more like a companion than a chatbot. The idea came around a real repetitive tasks, being organized, and asking everyday questions that could be simply answered.

What it does

Halo is a floating desktop widget (PyQt6 + OpenGL) that sits on screen and expands into a chat panel on click. It's backed by Claude, with tool access to:

• Google Calendar
• Web search
• Gmail drafting
• Session memory
• Local RAG

How we built it

Started with the shell, a system tray + always-on-top orb window using PyQt6, click handlers toggling a slide-out chat panel. Then wired Claude's streaming API for token-by-token responses so the UI feels alive instead of waiting on a spinner.

Google integrations (Calendar, Gmail) share one OAuth flow and one token.json, requesting combined scopes up front so we don't handle multiple credential files. Tool-calling is handled through a single dispatcher in the Claude service layer — each capability (search, draft, save template) is defined as a schema + handler function, and Claude decides when to invoke them mid-conversation, looping until it produces a final text response.

What we learned

• Tool-use loops need careful state handling, appending assistant tool-call turns and tool-result turns back into the message list correctly is easy to get wrong and breaks the conversation.
• Keeping side effects (like sending real email) reversible changes the whole design, drafting-only was the right call, since it lets the assistant act confidently without needing to be perfectly right every time.

Challenges we ran into

• Streaming + tool calls together — chaining live token streaming with a multi-step tool loop meant handling partial state carefully so the UI doesn't show broken or duplicated output.
• Balancing autonomy with safety — letting the assistant write emails on your behalf is powerful but risky, the border (drafts only) took multiple attempts to phrase correctly in the system prompt so Claude asks before acting instead of guessing.

Built With

• json
• pyopengl
• pyqt6
• python
• sql