MirrorMind

MirrorMind is an AI copilot that detects stress, confusion, and cognitive overload in real time, then intervenes instantly to restore focus before performance breaks down.

Comment

Inspiration

We started with a simple observation: people often realize they are overwhelmed only after their performance has already started to decline.

Whether someone is studying for an exam, giving a presentation, interviewing for a job, working through a difficult problem, or handling a high-pressure task, the warning signs often appear first as subtle changes in behaviour: rereading the same information, losing focus, becoming frustrated, rushing, disengaging, or repeatedly making the same mistake.

Yet most technology is completely blind to these signals.

Productivity tools measure what you accomplish. Chatbots wait for you to ask a question. Meditation apps require you to recognize that you need a break. Traditional software understands the task, but not the human performing it.

We wanted to build something fundamentally different.

What if your computer could recognize that you were struggling before you consciously realized it yourself?

That question became MirrorMind, an AI human-performance copilot designed to detect rising stress, confusion, cognitive overload, and disengagement in real time, then intervene before those states become performance failures.

Our inspiration was not to build another chatbot or another wellness app. We wanted to explore what happens when AI becomes aware of the human side of the interaction.

What it does

MirrorMind continuously analyzes multimodal signals from the user's interaction with their computer, including webcam-based physiological and behavioural signals, facial information, voice, typing behaviour, mouse activity, engagement, and contextual interaction patterns.

It then uses AI to reason across those signals and identify changes such as:

  • Rising stress
  • Cognitive overload
  • Loss of attention
  • Confusion
  • Frustration
  • Disengagement
  • Performance deterioration

The important part is that MirrorMind does not stop at detection.

It acts.

When the system identifies that a user may be struggling, Gemini analyzes the surrounding context to determine what could actually help. MirrorMind can then intervene through ElevenLabs with a natural voice interaction rather than another distracting notification.

For example, if a student has repeatedly attempted the same calculus problem while their engagement is declining and stress signals are increasing, MirrorMind can recognize the pattern and say:

"Looks like this concept is slowing you down. Want me to explain it differently?"

The user can then receive a simpler explanation or alternative way of approaching the problem.

MirrorMind also visualizes the user's state through a Mental State Orb, providing an intuitive real-time view of Focus, Stress, Confidence, Energy, and Engagement.

We also designed Cognitive Replay, which transforms an entire session into a concise narrative of the user's performance: when they were most focused, when stress increased, where they encountered difficulty, and how their state changed after intervention.

Instead of simply telling users what happened, MirrorMind attempts to change what happens next.

How we built it

MirrorMind is a multimodal, real-time AI system built by combining several technologies into one pipeline.

Presage forms the sensing layer. We use webcam-based signals to capture physiological and behavioural indicators such as heart-rate estimation, breathing-related signals, engagement, and facial information.

Gemini acts as the reasoning engine. Rather than treating each signal independently, Gemini receives the relevant context and helps determine what the combination of signals may indicate and what intervention is appropriate.

ElevenLabs powers the conversational intervention layer. We wanted MirrorMind to feel like a human-performance copilot rather than a notification system, so the AI can speak naturally to the user at the moment assistance is needed.

Tiger Data stores the evolving timeline of signals and interactions. This allows us to represent the user's state as a time series and power both live visualization and historical analysis.

Auth0 handles secure authentication and user identity, allowing users to maintain their own profiles and experiences.

Vultr provides the infrastructure for hosting the backend, APIs, and real-time components required to connect the system together.

The result is a pipeline where raw human signals become contextual understanding, contextual understanding becomes an intervention, and the outcome of that intervention becomes new data.

That closed feedback loop is what makes MirrorMind more than a passive analytics dashboard.

Challenges we ran into

One of our biggest challenges was turning noisy human signals into meaningful context.

There is no single measurement that can tell us whether someone is overwhelmed. A change in heart rate could mean stress, excitement, movement, or something completely unrelated. Reduced mouse activity could mean disengagement—or simply that someone is reading.

This meant we could not rely on individual signals. We had to think about patterns across multiple signals and changes over time.

Another major challenge was real-time responsiveness.

MirrorMind only feels useful if it can detect a change and respond quickly enough for the intervention to matter. Designing the pipeline so sensing, reasoning, data processing, and voice generation work together without creating a noticeable delay was a major engineering consideration.

We also had to solve the problem of over-intervention.

An AI that interrupts you every time it detects a small change in your behaviour would quickly become another source of cognitive overload. MirrorMind therefore needs to distinguish between a momentary fluctuation and a meaningful pattern that warrants intervention.

Finally, we faced a product-design challenge: how do you expose an extremely complex AI system without making the user interface extremely complex?

Our answer was to hide the complexity behind a simple experience. Instead of overwhelming users with dozens of metrics, we created the Mental State Orb and a small set of intuitive signals that communicate the user's state immediately.

Accomplishments that we're proud of

We are most proud that we moved beyond the typical "AI chatbot + dashboard" hackathon architecture.

MirrorMind creates a complete real-time interaction loop:

The system sees a signal → understands the context → makes a decision → speaks to the user → measures the response.

That makes the technology immediately visible during a demonstration.

We are also proud of the way we integrated multiple technologies into a single experience. Presage provides human-state signals, Gemini provides contextual reasoning, ElevenLabs provides natural voice interaction, Tiger Data provides the temporal data layer, Auth0 provides identity, and Vultr provides the infrastructure.

Most importantly, we are proud of the central product insight:

AI should not only understand what a user is doing. It should understand when the user is struggling to do it.

We built MirrorMind around that idea from the ground up.

What we learned

Our biggest lesson was that detection is not understanding.

Collecting more data does not automatically produce a better AI system. Human behaviour is noisy, contextual, and highly individual. Meaning comes from the relationship between signals, the user's activity, and how those signals change over time.

We also learned that the best intervention is not necessarily the most sophisticated one.

If a user is already cognitively overloaded, giving them another complicated interface can make the problem worse. Sometimes the most valuable AI interaction is a short sentence spoken at exactly the right moment.

We learned to think about AI less as a model that produces an answer and more as a closed-loop system that observes, acts, and learns from the result.

Most importantly, we learned that building technology around humans requires designing for the human first and the technology second.

What's next for MirrorMind

MirrorMind is currently a prototype, but the underlying concept can extend far beyond a student study environment.

Our next step is improving personalization. Rather than assuming the same signals mean the same thing for everyone, MirrorMind could learn each user's individual baseline and identify meaningful deviations from it.

We also want to expand Cognitive Replay into a long-term performance intelligence layer that helps users discover patterns across days and weeks—not just individual sessions.

From there, MirrorMind could support completely different environments:

  • Education: detect when students are stuck and adapt explanations in real time.
  • Interviews: identify rising anxiety and provide discreet preparation or grounding.
  • Presentations: detect escalating stress and help speakers regulate in real time.
  • Professional work: identify cognitive overload during demanding workflows.
  • Customer support: recognize escalating frustration or disengagement during difficult interactions.
  • High-pressure environments: provide contextual assistance when cognitive load becomes a critical factor.

Our long-term vision is to make MirrorMind an adaptive human-performance layer for computing.

Today, computers understand files, applications, commands, and workflows.

We believe the next generation should also understand something much more fundamental:

the human using them.

Built With

Share this project:

Updates

Submission history