Clippy!

Clippy AI — Make Clippy Alive!

An OS-level AI companion that actually understands you.

Vision

Clippy AI isn’t another chatbot.
It’s the foundation of an AI-native OS layer — where the model isn’t summoned by a prompt but always present, understanding your behavior, anticipating your needs, and helping you only when it truly matters.

Figma CEO Dylan Field once said in a YC podcast that “we’re still in the MS-DOS era of AI UX” — users have no natural way to explore what a model can do. That insight became our north star:
What if AI wasn’t a chat window, but part of your operating system?

Clippy AI is our answer to that question.

What Clippy AI Does

Clippy AI continuously observes your workflow across apps and responds intelligently at the right moment — not with pop-ups or interruptions, but as a silent, context-aware layer of assistance.

It currently features four specialized agents:

🐛 Debug Agent

Understands your development environment across IDEs, terminals, and browsers.
Detects and classifies errors (syntax, runtime, type, etc.) and provides real-time, context-aware fixes.

Example: For Type 'string' is not assignable to type 'number', Clippy suggests parseInt(userId) or updating the function signature to string | number.

📚 Learning Agent

Acts when you’re stuck reading technical content for too long.
Gives ELI5 explanations, translates jargon, and simplifies concepts — only when you actually need help.

Example: “Think of attention like a spotlight at a concert — focusing on key words instead of lighting the whole stage.”

Writing Coach (In Development)

Cross-app writing assistant that improves tone, clarity, and readability.
Helps overcome writer’s block and simplifies technical writing.

Research Agent (In Development)

Understands when you’re researching across tabs, then automatically fetches and summarizes recent papers, articles, and expert insights.

Architecture

Layer 1 – Classifier (GPT-4o Mini)
Runs every 7 s, cheap & fast (~$0.15 / 1 M tokens).
Classifies states: error / idle / writing / research / code / normal.
Layer 2 – Specialized Agents (Claude 3.5 + Bright Data)
Deep reasoning triggered only when needed (~20–30 % active time).
≈ 20× cheaper: ~$12 / month vs $200 +.

Stack: Electron · TypeScript · React · Tailwind · Framer Motion · SQLite · Tesseract.js (OCR)

Experimental — Temporal Memory Layer

Lives in our testing branch — not fully launched yet, but core to our long-term vision.

Current AIs understand the present, but forget the path that led there.
We wanted Clippy to remember context over time — what you changed, what caused a bug, and how your day unfolded.

Concept

Every few seconds, Clippy records semantic deltas: changes on screen, typed code, new errors.
These deltas are embedded and stored in ChromaDB, forming a temporal semantic memory.
Using vector search, Clippy can reason across time:

“You modified login.tsx three minutes before the type error appeared — that’s probably the cause.”

Beyond Debugging

Because every event has a timestamp, Clippy can also summarize your day:

“Today you spent 60 % debugging the auth flow and 40 % reading about Transformer attention.”

This memory layer refreshes daily — keeping summaries, not raw screenshots — preserving privacy and efficiency.
It’s the first step toward turning AI from an assistant into a working-memory system.

Key Engineering Wins

macOS Permission Lockdown: Custom onboarding + permission status detection.
JSON Parsing Chaos: Regex-based fallback parser for malformed AI responses.
Over-eager Suggestions: Sequential frame analysis (3–5 frames) + 80 % confidence threshold.
Cost Optimization: Text-first OCR, compressed screenshots, state-change triggering.

Why It Matters

This isn’t “AI that helps you code.”
It’s the beginning of AI as part of the operating system — an always-running intelligence layer that observes, understands, and assists based on continuous behavior.

The long-term vision is a “Cursor for the OS” — an ambient AI that operates beyond apps, reasons across sessions, and understands you over time.
Not reactive, but proactive.
Not a chatbot, but an evolving presence inside your workflow.