Sentri

Inspiration

We wanted to move beyond chat-based AI and build something that actually helps you get things done on your computer. Tools like assistants and copilots are powerful, but they often stop at suggestions.

Sentri was inspired by a simple idea:

What if your AI could see what you see and actually act on it?

What it does

Sentri is a desktop AI assistant that goes beyond conversation. It can:

🖥️ See your screen and explain what’s happening 🔍 Understand context from what you’re doing ⚡ Take actions like opening apps, managing files, and more 📄 Summarize PDFs and documents instantly 🧠 Remember useful context to improve over time

Instead of switching tabs and tools, you just ask—and Sentri handles it.

How we built it

Sentri is built as a modular AI system with:

🧩 A controller-based architecture for handling tools and actions 🧠 A custom AI “brain” pipeline with memory retrieval and context injection 🔌 Tool integrations like: File search & system control PDF summarization Screen understanding via vision models 🎨 A floating desktop UI with a mascot-style assistant for quick interaction ⚡ Threaded processing to keep the UI smooth and responsive

We focused on making everything feel instant, natural, and actionable.

Challenges we ran into

⚠️ Integrating vision with real-time interaction (screen capture + AI response) 🧵 Managing threading without freezing the UI 🧠 Designing a reliable memory system that prioritizes useful context 🔐 Handling OS-level automation limitations (especially on Linux/Wayland) 💸 Working around API limits and costs during development

Accomplishments that we're proud of

✅ Built a working context-aware AI assistant, not just a chatbot ✅ Implemented screen understanding + action execution ✅ Designed a custom memory system with weighted retrieval ✅ Created a smooth, interactive desktop assistant experience ✅ Shipped a functional product with real use cases

What we learned

🧠 AI becomes far more powerful when combined with tools and context 🎯 UX matters as much as intelligence—responsiveness is everything 🧩 Modular architecture makes it easier to scale and add capabilities ⚖️ There’s a constant tradeoff between power, cost, and performance 🚀 Shipping early is better than chasing perfection

What's next for Sentri

🗣️ Voice interaction for hands-free control 🤖 Smarter autonomous task execution 🌐 More integrations (browser, apps, workflows) 🧠 Improved long-term memory and personalization 💻 Polished UI and onboarding experience 🚀 Preparing for a wider public launch

Built With

• llama-cpp
• pyside6
• python