Breathe-PulseAi

Breathe-Pulse: Mindful Breaks, Intelligently Timed.

Comment

Inspiration

In today’s always‑on digital world, knowledge workers suffer from unseen fatigue, eye strain, and stress, often too late. We wanted a tool that doesn’t just remind you to take a break, but actually senses when you need one, and delivers it in a friendly, context‑aware way.

“What if your desktop could be an empathetic co‑pilot, nudging you toward better focus and well‑being before burnout sets in?”

That question drove us to build BreathePulse.

What it does

BreathePulse runs locally in your browser, leveraging your webcam (with explicit permission) to detect early signs of stress or eye strain. When it senses you’re due for a reset, it offers a 60‑second microbreak tailored to you:

🧘 Mindful Breathing: Guided inhale/exhale sessions.

🤸 Posture Resets: Quick stretches for neck, shoulders, and back.

🧩 Cognitive Refreshers: Fun mini‑puzzles to reboot your focus.

Each suggestion is delivered as a gentle notification you can accept or dismiss. Your feedback (“👍” or “👎”) helps BreathePulse learn which breaks work best for you, using simple reinforcement‑learning principles.

How we built it.

Under the Hood: Our Real‑time Adaptive Pipeline BreathePulse’s magic lives in three layers that all run quietly on your machine, no webcam data ever leaves your local device.

Computer Vision Layer (On‑Device Processing) Every time your webcam delivers a raw video frame, it enters a high‑performance on‑device pipeline built on MediaPipe Face Mesh (compiled to WebAssembly). We first detect and track dozens of facial landmarks, your eyes, brows, mouth, jawline, then compute four continuous metrics every 30 ms:

Eye Aspect Ratio (EAR) and blink frequency/duration to gauge eye fatigue. Facial Action Units (for instance, AU4 brow‑lowerer or AU7 lid‑tightener) to sense rising tension. Head Pose Estimation to flag static posture or fidgety micro‑adjustments.

These live values are aggregated into a time‑series feature vector that never leaves your machine, ensuring privacy-by-design.

Cognitive Engine (Fine‑Tuned LLM) That feature vector, together with your personal break history (what you accepted or dismissed, at what hour), is marshaled into a dynamic prompt for a compact, instruction‑following LLM. We distill a model like GPT‑4o‑mini into a 4‑bit‑quantized artifact and run it via llama.cpp in WebAssembly (with WebGPU accel where available). A typical prompt looks like this:

“EAR=0.19; AU4=0.42; preferredBreak=breathing; time=14:30. Generate a gentle, 60‑second guided breathing exercise in an empathetic tone.”

The LLM replies with a single, context‑aware coaching suggestion, no extra chatter, just the precise microbreak you need.

Personalization Layer (Reinforcement Learning) Finally, a lightweight Q‑learning agent living in a Web Worker treats each break type (breathing, posture, puzzle…) as an “arm” in a multi‑armed bandit. It buckets your stress and eye‑strain levels into discrete states (Low/Med/High), notes the time of day, and consults a persisted Q‑table in IndexedDB. When you accept a break, it awards +1; dismissals incur –0.5; sticking with work for 5 minutes posts an extra +2. Over sessions it learns which quick reset delivers the biggest payoff in each context.

Putting It All Together Frame in → CV layer extracts EAR, AUs, head pose.

Features → LLM crafts a single, empathetic prompt.

Prompt → User sees a 60‑second break suggestion.

Feedback → RL agent updates its Q‑table for next time.

Every bit of compute, CV, LLM inference, RL updates, happens on your CPU/GPU (or in a tiny Python backend if you prefer). No images, no raw video, just anonymized numbers and a smarter break coach that lives in your browser.

Tech Stack:

Frontend: React + TypeScript, Shadcn/ui components, Tailwind CSS, Zustand for state.

Backend (optional): FastAPI + Python (for any server‑side logic), Firebase Auth & Firestore for user profiles and feedback logs.

Hosting: Databutton for seamless dev→prod deployments.

Challenges we ran into

Lighting & Diversity: Ensuring CV models work reliably across different webcams, faces with glasses and lighting conditions.

Interrupt Timing: Balancing helpful nudges without becoming another distracting popup.

On‑device Learning: Implementing an efficient RL loop that updates break suggestions in real time without draining CPU/GPU.

Accomplishments that we're proud of

🚀 Fully client‑side CV pipeline that never sends raw video off your machine.

🤖 LLM‑driven coaching that feels genuinely empathetic, not robotic.

📈 Adaptive break suggestions that improve with each interaction.

🎨 A mindful, minimalist UI that blends seamlessly into your workflow.

What we learned

Real‑time CV in the browser is viable and performant with modern APIs (WebGL + WASM).

Fine‑tuning an LLM for emotional intelligence requires carefully curated prompt templates and example dialogues.

Lightweight RL can meaningfully personalize user experiences even with sparse feedback signals.

What's next for Brea-the-PulseAi

🔧 Robustness & Accessibility: Expand break library (eye‑exercise routines, gratitude micro‑journaling), add voice guidance, and optimize for mobile.

🌍 Cross‑Platform: Desktop native builds (Electron) & VS Code extension.

📊 Insights Dashboard: Visualize your stress patterns and break history over time.

🤝 Community Features: Share favorite mini‑puzzles or breathing sequences with other users.

Access the video demo: https://asu.zoom.us/rec/share/KP9fIqOSYa3PUgLfOXG-2ymyLYdkFyjMKgSl_854Aa9smToUCDO8sRhfPMNCZpG5.p4cDWQbV9DB4Z8Sb?startTime=1745169116000 Passcode: Lj3&s3!X

Team Ctrl + Alt + Elite — Aashir, Aditya, & Sumedha Thank you for checking out BreathePulse.

Built With

Share this project:

Updates