Inspiration
Personal AI becomes truly useful when it can understand our work, remember context, access our files and applications, and take actions on our behalf.
But that creates a serious problem: if the same AI that decides what should happen also decides what it is allowed to do, a hallucination, prompt injection, or overly confident decision can become a real action.
That led to the core idea behind SHARMAN:
Reasoning can be probabilistic. Authority cannot be.
SHARMAN separates intelligence from permission. NVIDIA Nemotron can understand a goal, reason about it, create a plan, and request tools, but SHARMAN independently decides whether those actions are allowed.
Nemotron provides the intelligence. SHARMAN controls the authority.
What it does
SHARMAN is a security-first Personal AI runtime that can assist with daily digital work without giving an AI unrestricted control over the user's environment.
It supports tasks, reminders, inbox intelligence, local files, Windows applications, MCP-based tools, personal workflows, and Home events.
A user can ask SHARMAN to plan the day, create reminders, locate files such as the latest downloaded ZIP, open approved applications, reason over personal context, and coordinate multiple tools.
For more complex requests, NVIDIA Nemotron 3.5 Lightning on Nebius acts as the reasoning layer.
However, requesting an action is not the same as being authorized to perform it.
For example, SHARMAN's Home runtime can process Ring Developer Platform events through:
Official Ring Event → Ring Adapter → HomeEvent → VisitorAgent → Permission Engine → Alexa/Home response
If a bounded VisitorAgent requests a sensitive capability such as door.unlock, SHARMAN independently evaluates the request.
Required authority: HIGH_RISK
VisitorAgent ceiling: LOW_RISK
Result: DENIED
The language model cannot change that decision.
How we built it
SHARMAN combines a native personal assistant, deterministic authorization, MCP tools, NVIDIA Nemotron through Nebius Token Factory, and external integrations behind controlled adapters.
The main AI flow is:
User Request
→ NVIDIA Nemotron on Nebius
→ Intent / Plan / Capability Request
→ SHARMAN Permission Engine
→ Approved MCP Tool
→ Result + Audit
NVIDIA Nemotron 3.5 Lightning is accessed through Nebius Token Factory and provides reasoning and planning.
The model can determine what tools might be useful, but authorization remains outside the model.
The project uses:
- NVIDIA Nemotron 3.5 Lightning
- Nebius Token Factory
- Model Context Protocol
- TypeScript
- Node.js
- Express
- React
- Rust
- Tauri
- SQLite
- Zod
- Windows APIs
- Gmail OAuth
- Ring Developer Platform
- HMAC-SHA256 webhook verification
- local/offline model support
For Home events, official Ring-format events pass through a dedicated adapter and are normalized into SHARMAN's internal HomeEvent format before being given to the bounded VisitorAgent.
The Ring path preserves duplicate suppression, source provenance, event filtering, and SHARMAN's existing permission boundaries.
Challenges we ran into
The hardest challenge was not connecting an LLM to tools. It was defining where the trust boundary should exist.
Agentic models are capable of choosing tools and generating multi-step plans, but allowing model output to directly become system authority creates a dangerous coupling between reasoning and privilege.
SHARMAN therefore treats model output as a request, never as authorization.
Another major challenge was connecting external event systems without weakening those guarantees.
For the Ring integration, we needed to preserve event provenance, validate HMAC-SHA256 signatures, suppress duplicate events, normalize external payloads into a controlled internal event format, and make sure the VisitorAgent could not gain additional authority simply because an external event arrived.
We also had to be careful about truthful demo provenance. Official Ring-format test events are clearly labeled as test events rather than being presented as physical hardware events.
Accomplishments that we're proud of
SHARMAN is more than a chatbot.
It includes a native local-first Windows assistant, persistent tasks and reminders, inbox intelligence, typed MCP tools, permission-aware execution, local storage, Home workflows, and auditable tool activity.
For this hackathon, we integrated NVIDIA Nemotron 3.5 Lightning through Nebius Token Factory as SHARMAN's reasoning and planning layer.
We also completed an end-to-end Ring Developer Platform event path:
Official Ring v1.1 event
→ verified Ring adapter
→ HomeEvent
→ bounded VisitorAgent
→ SHARMAN Permission Engine
→ controlled response
The Ring integration includes HMAC-SHA256 signature verification, event binding, duplicate suppression, non-human motion filtering, and preserved denial of high-risk actions.
The Ring E2E suite passes 8/8 checks, and the Home E2E scenarios also pass.
Most importantly, the integration did not require weakening the VisitorAgent or changing SHARMAN's permission boundary.
We are especially proud of this principle:
A smarter model should not automatically become a more privileged model.
What we learned
The biggest lesson was that Personal AI needs a different architecture from ordinary chat applications.
With a chatbot, a bad model response is usually just incorrect text.
With an agent connected to tools, a bad response can become a real action.
That means model accuracy alone is not enough.
We learned to separate:
What the AI believes should happen
from
What the system allows to happen.
We also learned that external events need the same discipline as AI-generated actions.
A Ring event, Gmail message, user request, or model-generated plan should not automatically bypass the permission system.
Everything is normalized into controlled inputs and evaluated through the same authority boundary.
What's next for SHARMAN
The next major step is turning SHARMAN into a complete personal agent operating layer.
Nemotron will be able to decompose larger goals and propose temporary specialist agents for tasks such as coding, research, project work, and planning.
Each temporary agent will receive only the tools and memory required for its task.
After completion:
- its result is returned,
- useful experience can be retained,
- temporary capabilities are revoked,
- and the temporary agent is terminated.
Persistent agents such as the Windows Agent and Daily Planner Agent can remain available for recurring workflows while operating under fixed capability ceilings.
SHARMAN will also expand its memory system so similar successful workflows can be retrieved and used when creating future agents.
When cloud access is available, Nemotron on Nebius can handle complex reasoning and orchestration. When offline, a local model can continue supporting basic personal-assistant functionality.
The long-term goal is:
A Personal AI that becomes more capable without automatically becoming more privileged.
Agents should not live forever, and neither should their permissions.
Built With
- amazon-bedrock
- amazon-ring-developer-playground
- express.js
- github
- gmail-api
- model-context-protocol-(mcp)
- nebius-token-factory
- node.js
- nvidia-nemotron-3.5-lightning
- oauth-2.0
- opentelemetry
- react
- rust
- sqlite
- tauri
- typescript
- windows-apis
- windows-credential-manager
- zod
Log in or sign up for Devpost to join the conversation.