I was inspired by the "Time" Youtube video with Hans Zimmer and Alan Walker that portrays a music event being synchronized across multiple locations: https://youtu.be/axRAL0BXNvw
I started thinking about the pandemic and overcrowded venues. How can you create a shared, live experience but keep people more separated in smaller venues? You can stream video of the performance to multiple locations but other "magical" elements of the show are lost in the translation.
The work of the lighting designer or "LD" is one of those key elements. Although it often goes unnoticed, stage lighting gives depth to an experience that communicates emotions and makes an event a magical experience.
What it does
LD Sync allows a single lighting designer or light board operator to control replicas of the same lighting setup across multiple locations distributed over a large metropolitan area covered by 5G networks. If you turn on the Red lamp in a particular lighting fixture will turn it on across all the locations that are connected. In addition, event attendees can scan a QR Code for a web app that turns their mobile device into a component of the light show.
LD Sync consists of four main components: The Console, the Controller, the Local Node, and the Audience App. Each component of LD Sync connects over a 5G carrier network so no hard-wiring is necessary. A venue can be created in the middle of a farm field provided the lighting is battery powered. 😀
LD Sync Console
The LD Sync Console is a web application that connects to the LD Sync Controller via WebSockets (TLS secured) over a 5G carrier connection for real-time, low-latency communication. The web interface looks similar to a physical DMX 512 lighting console and allows the lighting designer or light board operator to control light fixtures similar to traditional physical devices. The difference is that instead of one light fixture in one location listening for control signals there may be several fixtures listening across multiple locations.
LD Sync Controller
The central hub of the LD Sync System is the Controller Node.js service which resides on an AWS Wavelength EC2 t3.medium instance running Ubuntu 20 LTS. The controller provides an HTTPS web server and WebSocket server that the other three components connect to using HTTPS initially and then WebSockets. All connections are TLS encrypted with a LetsEncrypt certificate so the web browsers won't complain about the switch to WebSockets. The controller takes events (JSON) coming in from the console and broadcasts them to all the connected client components. The JSON events are simple objects with the DMX 512 channel and the new fader value.
LD Sync Local Node
There is one LD Sync Local Node for each location with a mirror lighting setup. The nodes are laptops with 5G network connections and an Enttec Open DMX device connected to a USB port. Each node runs a Node.js app that connects to the controller via a WebSocket connection and listens for new events. When an event arrives the Node.js app sends it to the Enttec Open DMX interface which is hardwired via DMX cabling to the physical light fixtures.
LD Sync Audience App
Concert attendees can scan a QR code to load a client app that connects to the controller via WebSockets. The audience app listens for events and changes the screen background color to match the lighting events sent. Fans can hold their phones up at specific points in the show to create a synchronized, crowd-sourced lighting fixture.
How I built it
Challenges I ran into
- I have to communicate with the Wavelength EC2 instance through a relay EC2 instance in the region. This complicated development and deployment a little.
- Setting up the required VPC and subnets was a little challenging.
- I couldn't set up LetsEncrypt directly on the Wavelength EC2 instance because LetsEncrypt couldn't use the ACME protocol to verify I controlled the machine. I set it up on the relay EC2 instance and temporarily assigned the DNS A record to the relay instance to get the LetsEncrypt certificate. I then transferred the cert to the Wavelength EC2 instance and moved the A record back.
- This was my first time setting up WebSockets over a TLS connection.
Accomplishments that I'm proud of
I created a new, live lighting control system and got it all to work. :)
What I learned
- How to set up and connect to a Wavelength EC2 instance.
- How to set up secure WebSocket connections.
- DMX 512 signaling and a lot of new lighting designer lingo: Like "LD", look, fixture, scene, chase, etc.
What's next for LD Sync
- Although I focused on lighting for this project, the LD Sync system could be used to control anything that needs a low-latency connection. I.e., MIDI, animatronics, cue video content, etc.
- I would like to add scene chases.
- Currently the console settings are stored in local storage on the browser. I would like to switch this to a session/server-side persistence mechanism that would follow the user across devices.
- Addition of "show" profiles so a single console app instance could be used for multiple shows with completely different lighting setups.
- The local node hardware could be a Raspberry Pi with some type of 5G radio "hat".
- A user interface to sequence the lighting scenes to music would be cool.
- It would be interesting to see how MQTT performs for communication between the controller and local node.
- It would be cool to encode DMX signals from a hardware lighting console and send them over the LD Sync System.
- Support for "pixel" lights.