Mantle

Inspiration

AI today gives answers, but not understanding.

Every AI copilot sees prompts — not behavior. It operates on isolated inputs instead of the full workflow across tools. As work becomes increasingly fragmented across email, browser tabs, Slack, CRMs, and internal dashboards, AI systems lack the persistent behavioral context needed to act intelligently.

We asked:

What if AI could observe how you actually work and build a structured understanding over time?

Mantle was inspired by the idea that AI needs a behavioral context layer beneath everything else — a foundation that models how work actually happens.

What It Does

Mantle is a local-first behavioral context engine.

It:

Captures desktop activity (screenshots + OCR)
Extracts structured entities (people, projects, topics, workflows)
Builds a temporal context graph of user behavior
Retrieves relevant past activity in real time
Generates high-precision next-action suggestions
Executes actions with one click

Instead of asking users what they want, Mantle predicts intent from behavioral patterns.

We model context as:

$$ G_t = (V_t, E_t) $$

Where:

V_t = entities (people, documents, topics, tools)
E_t = weighted relationships over time

Suggestions are generated as:

$$ S = f(C_{current}, R(G_t)) $$

Where:

C_{current} = live screen context
R(G_t) = retrieved relevant subgraph
S = ranked next-action suggestions

How We Built It

Architecture

Desktop Capture → OCR + Vision Parsing → Entity Extraction → Encrypted Local Context Graph → Semantic Retrieval → Suggestion Engine → One-Click Execution

Key Technical Components

Change Detection

To avoid redundant processing, we compute:

$$ \Delta = \frac{1}{n} \sum_{i=1}^{n} |p_i^{(t)} - p_i^{(t-1)}| $$

We only process frames when:

$$ \Delta > \theta $$

Entity Validation (3+ Co-occurrence Rule)

$$ w_{ij} = \begin{cases} 1 & \text{if co-occurrence}(i,j) \ge 3 \\ 0 & \text{otherwise} \end{cases} $$

Temporal Decay

To prevent graph bloat:

$$ w_{ij}(t) = w_{ij} \cdot e^{-\lambda \Delta t} $$

Similarity-Based Deduplication

Merge entities when:

$$ \cos(\vec{e_1}, \vec{e_2}) > \tau $$

All storage is encrypted and local-first by default.

Challenges We Ran Into

1. Pixels → Structure

Screens are noisy. UI chrome, timestamps, and dynamic layouts polluted early graphs. Extracting high-signal entities required strict filtering and spatial reasoning.