AssistAI - Measure Your Cognitive Engagement With AI

AI makes you faster. But is it making you sharper? AssistAI tracks how you think alongside AI, not just how you use it, so you stay the smartest part of the conversation.

Comment

Inspiration

We noticed something unsettling about our own AI habits. We'd open ChatGPT, paste the response into a doc without really reading it, and move on. It felt productive, but over time, something was slipping. We weren't thinking as hard. We weren't questioning as much. We were getting faster, but not sharper.

Every tool before AI offloaded memory or computation. AI offloads reasoning itself. And unlike a calculator or a search engine, it's so frictionless you don't even notice it happening. There's no moment of resistance where your brain has to kick in. That's what scared us. Not AI itself, but the quiet erosion of the habits that make us good thinkers in the first place.

We couldn't find a single tool that measured this. Plenty of apps track screen time or block distractions, but nothing that asks the deeper question: when you sit down with AI, are you actually thinking? That's why we built AssistAI.

What it does

AssistAI is a Chrome extension that runs silently in the background while you use AI tools. It doesn't read your prompts or responses. Instead, it watches how you behave: how long you pause before responding, how far you scroll into an AI response, whether you edit your prompts before sending, how often you copy-paste output directly, and whether you abandon responses before they finish.

These behavioral signals feed into a Gemini-powered analysis that produces three scores at the end of each session:

  • Cognitive Engagement measures the quality of your attention. Are you reading carefully, editing thoughtfully, and pausing to reflect?
  • AI Reliance measures how much you're leaning on AI output versus doing your own thinking. High copy-paste rates and low editing signal passive consumption.
  • Prompt Quality measures the craft of your questions. Are your prompts getting longer, more refined, and more intentional over time?

You also get a weekly streak tracker, a popup dashboard with score rings, and specific recommendations on what to improve. Everything runs locally. No data ever leaves your browser.

How we built it

The core is a Chrome extension built with vanilla JavaScript that injects content scripts into AI platforms (ChatGPT, Claude, Gemini, etc.). These scripts listen for behavioral events: scroll depth, keystroke timing, copy events, idle gaps, prompt submissions, and response completions. All of this data is stored entirely in Chrome's local storage, with no external servers involved.

For the analysis layer, we integrated the Gemini API. At the end of each session, the collected behavioral metrics are sent to Gemini with a carefully crafted prompt that asks it to evaluate cognitive engagement, AI reliance, and prompt quality on a 0-100 scale and return actionable feedback. The landing page is hosted on GitHub Pages.

We chose to keep the architecture intentionally simple: no accounts, no databases, no subscriptions. You clone the repo, add a free Gemini API key, and it just works.

Challenges we ran into

Privacy as a design constraint was hard. Our earliest prototypes analyzed prompt content to gauge quality, and the results were better. But we made the deliberate decision to never read what users type. That meant we had to find purely behavioral proxies for cognitive engagement, which required a lot of experimentation to get right. Scroll depth, idle timing, and edit frequency turned out to be surprisingly reliable signals, but finding that combination took iteration.

Cross-platform detection was tricky. ChatGPT, Claude, and Gemini all structure their DOMs differently, and they change frequently. Writing content scripts that reliably detect prompt submissions, response completions, and copy events across all three platforms meant a lot of careful selector work and fallback logic.

Calibrating the scoring model was more art than science. Getting Gemini to produce consistent, meaningful scores from raw behavioral data required a lot of prompt engineering. Early versions were either too generous (everyone got 80+) or too volatile (scores swung wildly between sessions). We landed on a structured prompt format with explicit rubrics and examples that produces stable, useful results.

Accomplishments that we're proud of

We're proud that we built something that works without compromising on privacy. It would have been easier and arguably more accurate to read prompt content, but we proved that behavioral signals alone are enough to produce meaningful insight. That constraint made the product better and more trustworthy.

We're also proud of the framing. There are a thousand tools that help you use AI more. We built one that helps you make sure AI isn't quietly making you less. That felt like an idea worth fighting for.

And honestly, we're proud that it changed our own behavior. After a week of using AssistAI, we started catching ourselves mid-copy-paste and thinking, "wait, did I actually read that?" That moment of self-awareness is exactly what we set out to create.

What we learned

The biggest lesson was that what you don't track matters as much as what you do. By refusing to collect prompts or responses, we were forced to think more creatively about what "engagement" actually looks like from the outside. It turns out your behavior tells a remarkably complete story about your thinking, even when no one can see your words.

We also learned that people have strong, immediate reactions to this idea. Almost everyone we showed it to had one of two responses: "I need this" or "I don't want to know." Both reactions told us we were onto something real.

On the technical side, we learned that the Gemini API is surprisingly good at structured behavioral analysis when you give it clear rubrics and constraints. Prompt engineering for evaluation is a very different skill from prompt engineering for generation, and we got a lot better at it through this project.

What's next for AssistAI - Measure Your Cognitive Engagement With AI

Longitudinal tracking. Right now, each session is scored independently. We want to build trend lines that show how your cognitive engagement, reliance, and prompt quality evolve over weeks and months, so you can see whether your thinking habits are improving or degrading.

Platform expansion. We're exploring support for AI-integrated IDEs like Cursor and Copilot in VS Code, where the stakes of cognitive offloading are arguably even higher. Developers who stop understanding the code their tools write are building on a foundation they can't maintain.

Team dashboards. We've had early interest from educators and team leads who want aggregated, anonymized views of engagement patterns across a class or engineering team, not to surveil anyone, but to spark better conversations about how their groups are integrating AI.

Research partnerships. The behavioral data AssistAI collects (locally and with consent) could be valuable to cognitive science researchers studying how AI changes human reasoning patterns. We'd love to collaborate with academic labs working on this question.

Ultimately, we believe the sharpest version of the future is one where humans and AI make each other better. AssistAI is a small tool in service of that big idea.

Share this project:

Updates