Stella’s system architecture
Start a conversation with Stella.
Stella transcribes your voice input.
Stella explains how it can help.
Hear Stella’s spoken responses.
Confirm before recording your screen.
Review and discuss workflow suggestions.
View saved workflows and their steps.

Inspiration

Our moms spend a lot of time on repetitive tasks. They want to use AI to make their work easier, but they don’t know where to start. And we realized this challenge, the gap between interest in AI and the ability to use it effectively, is common among elderly people.

What it does

As AI technology advances rapidly, many people struggle to keep up with its capabilities and discover how to apply it to their everyday work.

We're building an AI companion that observes how users work and proactively identifies opportunities to make their tasks easier with AI. Rather than requiring users to understand AI or identify use cases themselves, our companion takes the initiative to suggest practical AI-powered workflows and helps users build them through interactive conversations.

Beyond automation, our companion also helps users understand how AI works through top-down explanations, starting with practical use cases and offering deeper technical insights whenever users are curious.

We primarily target older workers who want to incorporate AI into their work but don't know where to start.

Our solution is built around two core principles:

Approachable AI: By proactively identifying and suggesting relevant AI use cases, we lower the barrier to adoption without requiring any prior AI knowledge.

Understandable AI: By providing intuitive, top-down explanations of AI-powered workflows, we help users understand what AI can do, how it works, and why it's useful.

How we built it

Stella intergrates:

Inference: Cartesia, Gemini, Jev
Client: Mac app /w SwiftUI, Cartesia's speech model
Server: Table, storage, document, server... /w Convex, memory /w Moss

Challenges we ran into

Building an agent that works across diverse scenarios was one of our biggest challenges. Every user's workflow is different, making it difficult to build a generalizable system that can reliably identify repetitive tasks and turn them into executable AI workflows. We went through numerous rounds of testing and iteration across different scenarios to improve both task identification and workflow generation.

Another major challenge was balancing intelligence, responsiveness, and user experience. We wanted Stella to feel like a patient, tech-savvy family member who guides you through unfamiliar technology, rather than a complicated AI tool. To achieve this, we worked on providing the agent with richer context while minimizing response latency, targeting responses within 1.5 seconds. Our goal was to make interactions with Stella feel natural, seamless, and supportive.

Accomplishments that we're proud of

We successfully built a PoC for a proactive AI agent that can infer users' needs and take action without explicit instructions.

Unlike traditional AI assistants that respond to specific requests, our agent tackles open-ended, underspecified tasks where the user's goals aren't clearly defined. At first, the vast space of possible user intentions made this challenge seem almost impossible.

But seeing Stella successfully handle scenarios inspired by our own moms' everyday struggles with technology made all the effort feel truly meaningful and rewarding!

What we learned

We gained valuable insights into tackling multi-step, multi-tool tasks under partial observability, where agents must reason about users' intentions and make decisions with incomplete information. We believe these open-ended, real-world challenges will become increasingly important as AI agents move beyond responding to explicit instructions toward proactively assisting people in their everyday lives.

What's next for

We plan to evaluate Stella across a broader range of realistic, diverse scenarios to uncover failure cases and make our system more robust, reliable, and adaptable to real-world workflows.

On the top of that, we also hope to expand Stella to other groups who face barriers to AI adoption, including children and people with limited technical experience. By adapting how Stella interacts with users and explains AI concepts based on their needs, we aim to make AI more approachable and understandable for everyone.

Ultimately, our vision is to make the benefits of AI accessible to anyone, regardless of their age or technical background.