Despite common streaming file formats (Ogg) supporting a massive amount of audio channels, uses other than surround-sound are seldom seen, and never (to my knowledge) in streaming applications. This app is a short and simple proof of concept that streaming and other media services can make use of multi-channeled audio while maintaining: 1) Small file sizes 2) Low CPU usage (& low battery drain) 3) High quality audio streams
What it Does
The app provides 14 channels of synced audio, all with variable volume. The UI contains 7 slide bars that control the volume for each pair of channels (Master track, Main Vox, Harmony Vox, Pads, Plucks, Commentary, and Alternate Mix). Users can adjust each stem/track individually, cutting up and experiencing playback of a song in new ways.
How I Built It
All mixing is done in Superpowered Audio SDK at the C level inside of the JNI, with the Java layer responsible for the file I/O and UI events.
Challenges I Encountered
Despite the project being directly targeted at the popular streaming container OGG, limited support for multi-channel OGG files in Java led me to separate the problem of channel automation into two tasks: 1) Whether the problem is technically possible (mixing and individually processing 10+ channels of audio simultaneously) and viable (high quality audio, low CPU and battery usage). 2) Parsing a multi-channel OGG file into channel components.
This implementation focuses exclusively on the possibility and viability of this type of product, and due to time constraints, takes shortcuts around an otherwise full implementation of the idea. Additionally, due to Android Studio's limited functionality while error-checking C/C++ code, I ran into a number of complications using Superpowered SDK. To minimize these errors, the code at the C++ level is written for simplicity, and could be vastly improved in terms of organization, memory usage, and speed.
What I'm Proud of
Even with the drawbacks and limitations mentioned above, the app runs with 0 added latency on top of android's Audio I/O, with low CPU usage. The app provides examples of what is possible with this sort of technology.
What's next for fReMaster?
The next steps would be a full implementation of OGG channel parsing, combined with the mixing and synced playback already present. In addition, a broader range of digital audio effects (reverb, compression, filter) would be simple to implement, adding more mastering elements to the individual stems and overall experience.