Neuro-Sync Live

Neuro-Sync Live: Sales & Business Brief Global Mental Performance Crisis**: 77%** of employees report experiencing burnout. Attention spans** have shrunk to 8 seconds (less than a goldfish).

Comment

Inspiration

In a world of constant distraction and digital noise, maintaining "Flow State" or finding deep relaxation is harder than ever. Existing focus apps rely on static playlists—they play the same "Focus Beats" whether you are stressed, tired, or hyper-alert.

We asked: What if your music player could see how you feel and adapt in real-time?

We wanted to build a "Living System"—a privacy-first AI that understands your bio-feedback (stress, fatigue, focus) and dynamically tunes your brainwaves using scientifically proven binaural beats.

What it does

Neuro-Sync Live is a Progressive Web App (PWA) that transforms your device into a personal cognitive performance coach.

  1. Analyses You: It uses your webcam to detect subtle emotional cues, fatigue levels, and attention spans.
  2. Hybrid Intelligence:
    • Cloud Mode: Uses Gemini 1.5 Flash for deep, nuanced multimodal analysis of your "vibe" and environment.
    • Local Mode: Uses Gemma 2B (WebGPU) for completely private, offline, latency-free analysis directly in your browser.
  3. Neuro-Entrainment: It generates real-time Binaural Beats (Alpha, Theta, Gamma waves) and layers them under functional music. If you look stressed, it shifts to 10Hz Alpha waves to calm you. If you look tired, it shifts to 40Hz Gamma waves to wake you up.

How we built it

  • Frontend: Next.js 14 with Tailwind CSS v4 and Framer Motion for a premium, glassmorphic UI.
  • Cloud AI: Google Generative AI SDK (Gemini 1.5 Flash) running on Vercel Edge Functions.
  • Local AI: We integrated MediaPipe Tasks with Gemma 2B (Quantized) running on WebGPU. This allows a full LLM to run inside the browser without a backend.
  • Audio Engine: A custom TypeScript audio engine using the Web Audio API. It uses raw oscillators to generate precise binaural frequencies (e.g., Left Ear: 200Hz, Right Ear: 210Hz -> 10Hz Beat) and mixes them with ambient loops.

Challenges we ran into

  • WebGPU on Mobile: Getting the 1.3GB Gemma model to load and run efficiently on mobile browsers was a massive hurdle. We had to implement strict memory management and "ready state" checks to prevent Safari from crashing.
  • Audio Artifacts: Calculating precise frequency offsets for binaural beats while fading between tracks created "popping" sounds. We solved this with smooth gain nodes and exponential ramping.
  • Privacy vs. Power: Balancing the power of Gemini (Cloud) with the privacy of Gemma (Local). We built a "Hybrid Toggle" that lets the user choose their trade-off instantly.

Accomplishments that we're proud of

  • True Hybrid AI: We successfully built a system that seamlessly switches between a Cloud Giant (Gemini) and a Local Expert (Gemma) with a single click.
  • Real-Time Latency: Achieving a feedback loop where the music changes within seconds of your facial expression changing.
  • Privacy-First Design: Proving that powerful AI apps don't need to send your face to a server to work.
  • Patent Pending: We developed a unique method for bio-adaptive audio entrainment and have filed a UK patent application for the architecture.

What we learned

  • The Power of Small Models: Gemma 2B is surprisingly capable at specific tasks (like emotion classification) when prompted correctly, even running on a phone.
  • Browser Capabilities: modern Web APIs (WebGPU, Web Audio) are now powerful enough to replace native desktop applications for complex creative tools.
  • Neuro-Science: We learned deeply about brainwave frequency following responses (FFR) and how audio can directly influence cognitive states.

What's next for Neuro-Sync Live

  • EEG Integration: Connecting to consumer EEG headbands (Muse, Neurosity) for "True" bio-feedback beyond just visual cues.
  • Personalised Models: Fine-tuning Gemma on the user's specific facial cues over time to create a "Personal Ghost" that knows you better than anyone.
  • VR Port: Bringing the experience to Meta Quest / Apple Vision Pro for immersive neuro-meditation.

Built With

Share this project:

Updates