Audiotool OSC Integration for Live Performance

Integrate OSC into Audiotool and allow it to send/receive OSC messages over UDP, making Audiotool more suitable for live performance.

Comment

Inspiration

The inspiration for this project comes from a resurgence of audiovisual tool-building and performance in the creative technology scene. While Audiotool already facilitates real-time collaboration, there was no way to extend it's use past being a standalone DAW. We wanted to incorporate Audiotool into the ever-growing live audiovisual performance toolkit.

What it does

This project creates a bidirectional OSC (Open Sound Control) bridge between Audiotool and external applications such as TouchDesigner, Max/MSP, and TouchOSC. This bidirectionality enables:

  • Audiotool as an OSC Sender: Changes to mixer, effect, project, and device parameters in Audiotool get sent as OSC messages that external applications can receive. For example, changing the pattern in a Tonematrix changes which visual appears on screen.
  • Audiotool as an OSC Receiver: External applications can send OSC messages that modify these parameters once Audiotool receives them. For example, hand gestures modify reverb and delay in Audiotool.

With OSC capability, Audiotool transforms from a standalone DAW into a tool capable of being used for live performance.

How we built it

The architecture consists of two main components:

  1. Node.js OSC Bridge: Implemented via the osc-js library, this bridge handles bidirectional communication between our Node app and OSC-enabled software.

  2. OSC Message Handling: Mappings were implemented with reference to Audiotool's Nexus SDK to send and receive OSC messages.

Challenges we ran into

  • Real-time Performance: Unfamiliarity with Audiotool's event transactions led to increased latency, dropped commands, and other unexpected issues when sending and receiving OSC messages, especially when modifying multiple parameters at once.

What's next

  • Extended OSC mappings: Map additional Audiotool devices and parameters, including synthesizer controls, automation curves, and track-level effects
  • Improving Real-time Performance: Implement message batching and better transaction handling to mitigate any real-time hiccups.

Built With

Share this project:

Updates