SHARMAN

A permission-aware AI assistant where models can plan and act but cannot grant themselves authority.

Comment

Inspiration

AI assistants are becoming capable of reading our emails, using tools, interacting with services, and taking actions on our behalf.

But intelligence and authority are not the same thing.

An AI agent may make a perfectly reasonable decision and still attempt something it should never be allowed to do. A home assistant may recognize an expected delivery and conclude that unlocking the door would be helpful. A productivity assistant may decide to send an email when the user only wanted a draft.

The problem is not always malicious AI.

Sometimes the AI is simply wrong, overconfident, manipulated, or too proactive.

That led to the central idea behind SHARMAN:

AI can propose an action, but it cannot grant itself authority.

SHARMAN is a local-first, security-first personal AI runtime for Windows, built around bounded, permission-aware workflows.

Instead of giving one permanently powerful AI access to a user's entire digital life, SHARMAN gives task-specific workflows only the capabilities they need, for only as long as they need them.

The goal is to make personal AI:

local, scoped, temporary, auditable, and revocable.

What it does

SHARMAN connects personal productivity, conversational AI, smart-home context, and external services through one permission-aware runtime.

It currently includes:

  • Local tasks and reminders
  • Windows notifications
  • Read-only Gmail intelligence
  • Action and deadline extraction from emails
  • Daily planning across inbox actions, tasks, and reminders
  • Real Streamable HTTP MCP
  • 11 typed MCP tools
  • An Alexa+ style MCP experience
  • Official Ring Developer Playground integration
  • Live Ring device discovery, status, and event-history access
  • Bounded Home workflows
  • Permission ceilings for task-specific workflows
  • Persistent results and audit traces
  • Optional Amazon Bedrock / Nova explanations
  • Public AWS-hosted demonstration

The important part is not simply that SHARMAN can connect to these services.

It is how access to them is controlled.

Personal AI without permanent authority

SHARMAN is designed as a personal AI runtime rather than a permanently privileged cloud assistant.

Sensitive integrations remain local wherever possible.

For example, the native Windows application connects to Gmail using read-only OAuth, while credentials are kept in Windows Credential Manager instead of being exposed through the public web application.

When SHARMAN needs to perform a task, it can run a bounded specialist workflow:

User request
      ↓
SHARMAN Supervisor
      ↓
Task-specific workflow
      ↓
Minimum required capabilities
      ↓
Permission checks
      ↓
Tool execution
      ↓
Result + audit
      ↓
Workflow ends

Supporting many integrations does not mean every workflow gets access to all of them.

One workflow may read reminders but not email.

Another may inspect Home context but not control a lock.

Another may analyze inbox actions without being able to send messages.

Instead of giving one AI permanent access to your digital life, SHARMAN gives bounded workflows the minimum authority they need for the task.

Permission-aware execution

Every sensitive action passes through deterministic permission checks.

For example, SHARMAN's Home demonstration includes a bounded VisitorAgent.

It can:

✓ read Ring/Home context
✓ inspect expected-visit information
✓ prepare an Alexa response
✓ propose reminders

But it cannot:

✗ unlock a door
✗ execute arbitrary system commands
✗ make purchases

If the VisitorAgent requests:

Requested capability: door.unlock
Required authority:   HIGH_RISK
VisitorAgent ceiling: LOW_RISK

SHARMAN returns:

DENIED

The important point is that the model does not own the security boundary.

Even if an AI concludes that an action would be useful, it cannot increase its own authority.

A smarter agent should not automatically become a more privileged agent.

How we built it

The native SHARMAN application is built with:

  • Rust
  • Tauri 2
  • React
  • TypeScript
  • SQLite
  • Windows Credential Manager
  • Windows notifications

The web and MCP runtime uses:

  • Node.js
  • TypeScript
  • Express
  • Model Context Protocol SDK
  • Zod
  • React
  • Vite

Amazon technologies used include:

  • AWS ECS
  • Amazon Bedrock
  • Amazon Nova
  • Ring Developer Platform
  • Alexa+ style MCP integration

The core execution model is:

User / native app / web / Alexa+ style client
                         ↓
                  SHARMAN Supervisor
                         ↓
                Bounded workflow
                         ↓
            Deterministic permissions
                         ↓
                  Permitted tool
                         ↓
               Result + audit

Home information follows a similar path:

Ring API / supported event / controlled simulation
                         ↓
                  Parse + normalize
                         ↓
                     HomeEvent
                         ↓
                 Bounded workflow
                         ↓
               Permission decision
                         ↓
                   Result + audit

The AI layer can reason about context.

SHARMAN decides what can actually happen.

Alexa+ and MCP

SHARMAN implements a real Streamable HTTP Model Context Protocol server.

The current MCP runtime exposes 11 typed tools covering tasks, reminders, inbox actions, assistant requests, Home state, Ring events, and Bedrock explanations.

The Alexa+ style web experience acts as a real MCP client.

For example, when the user asks:

“What tasks do I have today?”

the interface sends a real MCP tools/call request.

The demo visibly shows:

MCP request
      ↓
Selected tool
      ↓
Permission level
      ↓
Execution result

Alexa+ acts as the conversational surface.

SHARMAN remains the execution and authority layer.

Alexa+ is how the user talks to SHARMAN. SHARMAN decides what is allowed to happen.

Gmail Inbox Intelligence

The native Windows application connects to Gmail using read-only OAuth with PKCE and a loopback callback.

SHARMAN can identify:

  • Actionable messages
  • Assessments
  • Interviews
  • Application updates
  • Follow-ups
  • Deadlines
  • Recent messages
  • Unread messages

The integration is intentionally read-only.

SHARMAN does not send, delete, archive, label, or modify email.

OAuth revocation and fresh reconnection were also tested so SHARMAN does not silently continue using a revoked authorization.

This follows the same principle:

Give AI the minimum capability required for the task instead of unrestricted account access.

The public web demonstration intentionally uses isolated demo data, while private Gmail data, native tasks, reminders, and Windows notifications remain inside the user's Windows runtime.

Ring Developer Platform integration

SHARMAN integrates with the official Ring Developer Playground.

The Ring client implements:

listDevices()
getDeviceStatus(deviceId)
getEventHistory(deviceId)

Live validation successfully confirmed:

  • Ring device discovery
  • Device status
  • Event history

The SHARMAN Home interface can display:

  • Official API connection status
  • Device online/offline state
  • Last reported time
  • Latest Ring event type
  • Event history
  • Source and provenance

Ring information is normalized while preserving its original event type, timestamp, device identity, and provenance.

This matters because an AI system should not invent certainty.

For example, a Ring on_demand live-view session remains an on_demand session.

SHARMAN does not turn it into a person, package, visitor, or threat unless the source information actually supports that conclusion.

The real Ring integration and SHARMAN's controlled Home safety simulations are intentionally kept separate.

Smart Home example

Consider an expected delivery.

A temporary VisitorAgent can inspect Home context, check an expected-delivery window, prepare an Alexa response, and request further action.

But it cannot directly unlock the door.

A future delivery workflow could look like:

Delivery information
        ↓
Temporary DeliveryAgent
        ↓
Expected delivery context
        ↓
Delivery tracking
        ↓
Alexa+ updates
        ↓
Access request
        ↓
Deterministic verification
        ↓
One-time gate access

The AI workflow would still not directly own the lock.

A separate deterministic AccessController could verify factors such as a delivery credential, OTP, passkey, time window, or user policy before issuing tightly scoped one-time access.

The separation is:

AI understands and requests.
Policy authorizes.
A deterministic controller executes.

Amazon Bedrock

SHARMAN optionally uses Amazon Bedrock with Nova 2 Lite as an explanation layer.

Bedrock does not determine permissions.

The deterministic permission decision happens first.

For example:

door.unlock
      ↓
Required: HIGH_RISK
VisitorAgent: LOW_RISK
      ↓
DENIED

Only after that result is fixed can Bedrock generate a natural-language explanation.

Bedrock may explain why an action was blocked, but it cannot turn:

DENIED

into:

ALLOWED

If Bedrock is disabled or unavailable, SHARMAN can fall back to a deterministic local explanation.

This means the safety boundary does not depend on cloud inference.

Bedrock explains the decision; it does not make the decision.

And:

Reasoning can be probabilistic. Authority cannot be.

Challenges we ran into

Separating real integrations from simulations

One of the biggest challenges was ensuring that SHARMAN never presented simulated or incomplete information as a real external event.

Ring Developer Playground live-view activity can appear in Event History as on_demand.

It would have been easy to interpret that as motion or a visitor.

Instead, SHARMAN preserves the actual source event type and keeps real Ring data separate from controlled SHARMAN safety simulations.

Keeping personal information private

The public demonstration should never expose the developer's Gmail account, OAuth credentials, local SQLite databases, or Windows notification data.

We therefore separated the public judge-facing environment from the private native runtime.

The web experience demonstrates the real architecture while personal integrations remain local.

Designing agents with limited authority

It is easy to build one agent and give it every available tool.

SHARMAN deliberately does the opposite.

Capabilities are explicit and workflows receive only the permissions they require.

This makes failures easier to contain and makes system behavior more predictable.

OAuth lifecycle and recovery

Authentication does not end after the first successful login.

SHARMAN also had to handle authorization revocation, stale credentials, fresh consent, and reconnection without silently substituting sample data for unavailable live information.

Optional cloud reasoning

Bedrock needed to remain optional.

SHARMAN therefore makes the deterministic decision first and validates any generated explanation afterward.

Cloud failure must never weaken the permission boundary.

Accomplishments that we're proud of

SHARMAN currently includes:

  • A working native Windows personal assistant
  • Real Streamable HTTP MCP
  • 11 typed MCP tools
  • Alexa+ style MCP experience
  • Read-only Gmail intelligence
  • OAuth revoke/reconnect handling
  • Local tasks and reminders
  • Visible Windows notifications
  • Bounded Daily Planner workflows
  • Bounded VisitorAgent workflows
  • Deterministic capability ceilings
  • Persistent audit traces
  • Live official Ring Developer Playground API validation
  • Ring device discovery
  • Ring device status
  • Ring Event History
  • Optional live-validated Amazon Bedrock / Nova explanations
  • Local deterministic explanation fallback
  • Public AWS ECS deployment
  • Native, MCP, Ring, Home, browser, and production-build validation

But the accomplishment we are most proud of is the architecture underneath those features:

The AI can request an action, but it cannot grant itself the permission required to execute it.

What we learned

The biggest lesson from SHARMAN was that AI safety is not only a prompting problem.

A model can be:

  • Wrong
  • Overconfident
  • Manipulated
  • Confused
  • Too proactive
  • Operating with incomplete context

The surrounding system should assume these situations can happen.

Instead of relying entirely on the model to behave correctly, SHARMAN places the authority boundary outside the model.

We also learned how important provenance is.

SHARMAN distinguishes between information originating from:

  • A real API response
  • A captured response
  • A documentation-derived test fixture
  • A controlled simulation
  • A deterministic workflow
  • An AI-generated explanation

This makes the system easier to debug, safer to operate, and easier to trust.

What's next for SHARMAN

The next step is to make SHARMAN's bounded workflow model richer while preserving the same authority boundaries.

Future workflows could include:

  • Temporary delivery agents
  • Verified one-time visitor access
  • Elder-care check-ins
  • Night Guard workflows
  • Away-from-home monitoring
  • Calendar-aware planning
  • Background research workflows
  • Secure remote household coordination
  • Passkey, NFC, or OTP-based access verification

Imagine ordering food or medicine.

SHARMAN could identify an expected delivery, create a temporary workflow only for that delivery, monitor its progress, notify the user through Alexa+, and request one-time gate access when the verified delivery reaches the home.

After the task is complete, the temporary workflow and its delegated authority end.

That captures the long-term direction of SHARMAN:

Autonomy should be scoped, temporary, auditable, and revocable.

SHARMAN is not about making AI less capable.

It is about creating an environment where increasingly capable AI can be trusted with increasingly useful tasks without automatically receiving unlimited authority.

SHARMAN

AI can be autonomous without being unrestricted.

Built With

Share this project:

Updates

Submission history