SpacialSync-Audio

Turn any room of smartphones into a perfectly synchronized, distributed spatial audio system. Built with React, WebSockets, and Web Audio API.

Comment

SpacialSync Audio Orchestra

Turn any room of smartphones into a perfectly synchronized, distributed spatial audio system.

SpacialSync is a real-time web application that links multiple mobile devices into a unified audio array. One device acts as the Conductor, controlling exactly what every connected Client plays and when—down to the exact millisecond.

Whether you're streaming an uploaded audio track to twenty phones at once, or generating ambient music on the fly, SpacialSync ensures zero network latency drift.

✨ Core Features

  • ⏱ Sub-millisecond Time Sync: Uses a custom Precision Time Protocol (PTP) handshake over WebSockets to calculate exact network latency and clock drift for every device.
  • 🎵 Synchronized Track Streaming: Upload any MP3/WAV file from the Conductor dashboard, or provide a direct URL. The server proxies the audio to avoid CORS, and schedules playback on all devices at an exact future timestamp using the AudioContext.
  • 🎹 Generative MIDI Engine: Don't have a track? The server runs a generative Fibonacci-based sequencer that streams continuous, ambient MIDI-style notes to all phones.
  • 🗺 3D Spatial Mapping: The Conductor interface features a live Three.js visualization. Every connected phone appears as a glowing sphere in 3D space, positioned automatically using their real-world GPS coordinates.

🏗 System Architecture

True multi-device synchronization is notoriously difficult on the web due to CORS restrictions, browser autoplay policies, and network jitter. SpacialSync solves this via a robust multi-step pipeline:

  1. Proxy & Preload: When the Conductor uploads an audio track, the server broadcasts the link. The server proxies external URLs so clients can bypass strict Web Audio CORS policies. Every phone downloads and decodes the AudioBuffer in the background.
  2. Future Scheduling: When the Conductor presses play, the server broadcasts a startAt timestamp exactly 800ms in the future.
  3. Execution: Each client calculates its execution time dynamically: javascript audioContext.currentTime + (serverStartTimestamp - Date.now() + networkOffset) / 1000 This ensures every device fires its BufferSourceNode at the exact same physical moment, completely erasing network latency.

🚀 Quick Start

Prerequisites

  • Node.js 18+
  • npm 9+
  • OpenSSL (pre-installed on macOS/Linux, required for generating certificates)

1. Generate SSL Certificates

The Geolocation and DeviceOrientation APIs strictly require HTTPS. Generate your local certificates by running:

bash generate-certs.sh

Note: This script will auto-detect your local IP address and print it to the console. Save this IP, you'll need it to connect your phones!

2. Install & Build

# Install server dependencies
cd server && npm install && cd ..

# Install and build the React client
cd client && npm install && npm run build && cd ..

3. Start the Server

cd server && npm start

The server will now be running on port 3443 (HTTPS).

📱 Connecting Devices

Make sure all devices (your computer and your phones) are connected to the same Wi-Fi network.

Role URL Route Description
Client https://<YOUR_LAN_IP>:3443/ (Default route) Open this on phones. Displays the pulsing sync UI and acts as a speaker.
Conductor https://localhost:3443/?role=conductor Open this on your computer. Displays the DAW dashboard, 3D map, and audio upload controls.

Handling the SSL Warning

Because the certificates are self-signed for local development, your phone's browser will warn you that the connection is not private.

  • Chrome: Click Advanced → Proceed.
  • Safari: Tap Show Details → visit this website.

Starting Audio

On the client view, tap the big Tap to start orb. Mobile browsers strictly require a physical user gesture before they allow the Web Audio API to play sound. You won't see any "Stop" buttons on the phone—all playback control is handled remotely by the Conductor dashboard.

🛠 Development Mode

If you want to modify the React code with hot-module reloading:

# Terminal 1: Start the backend server
cd server && npm start

# Terminal 2: Start the Vite dev server
cd client && npm run dev

You can then access the conductor interface at http://localhost:5173/?role=conductor.

(Note: The Geolocation API will not work over plain HTTP in development mode. To test GPS mapping, you must use the production build over HTTPS).

📄 License

MIT

👨‍💻 Made by

Sameer Abrar, Flexcrit Inc.

Share this project:

Updates