companion

Inspiration

Exam season is brutal. Students pull long hours, skip breaks, and push through exhaustion — not because they want to, but because they don't notice how depleted they've become until it's too late. We wanted to build something that could spot the warning signs earlier than the student themselves, and intervene gently before stress turns into burnout.

What it does

Companion is a desktop app that runs quietly in the background during study sessions and monitors three passive behavioural signals: keystroke rhythm, mouse movement patterns, and webcam snapshots. Every few minutes, it sends those signals to Claude, which assesses the student's current stress and fatigue level (LOW / MEDIUM / HIGH). If it detects signs of strain, it surfaces a non-intrusive intervention — a guided 4-7-8 breathing exercise, a 5-minute break with a countdown timer, or a gentle check-in prompt. A small floating HUD in the corner of the screen shows the current status without getting in the way.

How we built it

• Python + tkinter for the desktop UI and floating HUD
• pynput for passive keystroke and mouse monitoring (no content is logged — only timing and movement statistics)
  
• OpenCV for periodic webcam frame capture
• Claude API (claude-sonnet) as the reasoning engine — we send a structured behavioural summary plus an optional webcam image and ask Claude to assess stress level and recommend an intervention type
  • Multithreaded architecture: monitors run as background threads, analysis runs on a timed loop, and all UI updates are safely marshalled back to the tkinter main thread.

Challenges we ran into

• Privacy vs. signal quality: Webcam frames are the richest signal but the most sensitive. We spent time getting the consent flow right and making sure the app is transparent about what it captures.
• Avoiding false positives: A low typing rate could mean deep focus, not fatigue. We had to craft the Claude prompt carefully so it reasons holistically across all signals rather than over-indexing on any single one.
• tkinter threading: tkinter is single-threaded. Getting background analysis results to safely update the UI
• without crashes or race conditions — required routing everything through after() callbacks on the main thread.

Accomplishments that we're proud of

• A fully working multimodal stress detector that runs locally with no setup beyond an API key
  
• Claude's reasoning is shown directly to the user ("What I noticed:") — making the AI's assessment transparent and trustworthy rather than a black box
  
• The intervention UX feels genuinely calm and supportive, not naggy or alarming

What we learned

• Behavioural biometrics (typing cadence, mouse jitter) carry surprisingly rich signal about cognitive state — no wearable required
• Prompt design matters enormously for structured output: getting Claude to return consistent, parseable JSON while still sounding empathetic took several iterations
  
• Small UX decisions — modal vs. non-modal, wording of the intervention copy, which emoji to use — have a big impact on whether the app feels helpful or intrusive

Built With

• claud
• opencv
• pynput
• python
• threading
• tkinter