Aficionado

• 

  Aficionado logo

Inspiration

In 2011, I was a postdoc in the Centre for Digital Music at Queen Mary University of London. The project I was working on, LinkedBrainz, created a set of mappings from the MusicBrainz database to a graph representation, made available as open data. One application of this dataset was in the creation of music recommender systems, but it has so much potential beyond that.

Later, in 2016, I was working in a data science team at Elsevier. One of our most ambitious projects was to detect communities of science by deriving partitions within the graph of co-author relationships found in academic publications. Having found difficulty scaling this solution out, we hypothesised that the parallelisation offered by NVIDIA GPUs would speed the (eigenvector-based) calculation. Due to the lack of library support, I spent months writing CUDA kernels by hand.

In 2019, I was at NVIDIA GTC when I met Bradley Rees, who told me that all of that manual work could now be simply achieved using their cuGraph library. I wasn't bitter.

Since then, I've often thought about the combination of these three things. In music, and especially in jazz (as well as classical music and opera), aficionados pay a great deal of attention to the personnel on recordings — not just band leaders, but the whole ensemble, including session players.

What if we viewed playing on a recording together as a form of co-authorship? Are there insights to be gained from community detection in such a graph? And what if one could have a conversation with an AI agent with these insights, rather than rely on passive music recommendation?

What it does

Aficionado is not your run-of-the-mill recommender system. It's an AI agent with encyclopedic knowledge of recorded music, with which you can have an informed conversation about your favourite musicians and their collaborators and peers. It has a unique insight into communities of musicians who have worked together over the decades.

In broader terms, it's a potential future for user experience of streaming music services. Spotify, Tidal, etc., still rely heavily on social recommender systems. These can be very useful if you're into contemporary popular music — if you play an Ariana Grande track, you might be happy to hear Taylor Swift next — but they provide very little functionality to explore the rich connections between artists and their various recordings (which in jazz might be dozens of versions of the same piece with many different rosters of players). The best way to interrogate these connections is conversationally and we should be enabled to talk to an expert, just like we did when visiting record shops in the days before the Internet.

How we built it

Aficionado is based on a core dataset of metadata for recordings and musician credits from the MusicBrainz dataset. The MusicBrainz data dump is reconstituted in postgresql, then this derived dataset is extracted using SQL, transformed into graphs in NetworkX, and loaded into ArangoDB using nx-arangodb. Although MusicBrainz is an incredible resource — with more than a dozen different entity types, including 2.5m artists and 34.5m recordings — it is very complicated to query (as can be seen in the relatively simple task of extracting a graph of humans with shared credit on recordings). A graph database with a more natural representation and querying style makes it much simpler to interact with.

The next step in giving Aficionado its unique capabilities is to use cuGraph to carry out community detection within this graph. This is a complex and computationally-intensive process, but made easy by the cuGraph library of algorithms and their highly-parallelised implementation via GPGPU. This community information is added to the existing graph in ArangoDB.

The final step in bringing Aficionado to life is the use of langchain to effect AQL queries over the resulting graph, based on user intent. This is only possible because of the simple graph-based representation. Carrying out text-to-SQL over a schema as complex as MusicBrainz is all but impossible, even without the additional information computed with cuGraph.

Challenges we ran into

A first stumbling block was the lack of ARM64 support in the nx-arangodb library, which prevented prototyping on a Macbook. In typical fashion, python/pip gave very opaque errors but we soldiered on and created a solution, building the phenolrs dependency from source using maturin.

A second problem was that the existing project to reconstitute the MusicBrainz postgres database no longer works due both to the standard postgres v9 docker build's dependency on a broken Debian link, and also by the more-recent schema evolution in the database itself.

Accomplishments that we're proud of

Having addressed both of these challenges, we created new container builds so that others can take easy advantage of our fixes, bringing both databases and a properly configured CUDA environment up using Docker Compose.

What we learned

Everything is easier the second time around! Creating a graph representation of useful parts of the MusicBrainz dataset, and carrying out community detection with GPU acceleration, are made much easier using ArangoDB's integration of NetworkX and cuGraph.

What's next for Aficionado

While Jazz — and its culture of rich recording credits — makes a good test case for interfacing musical knowledge with user exploration via AI agents, other genres of music lend themselves to different graph-based possibilities. Labels ("who got their start on 4AD"), side projects ("what other bands did Kim Deal's bandmates play in after she left The Pixies?") and producer credits ("which other albums did Steve Albini engineer?") are standard fare for indie music fans, but difficult to explore on current streaming services. Often these connections are found via online fan communities (or Wikipedia) outside of these services.

Other means to find new musical connections ("for whose gigs did Nirvana play as the support act before they got signed to Geffen?"; "who's played at a festival obscure enough to have both Melt Banana and Pigs Pigs Pigs Pigs Pigs Pigs Pigs on the bill?") also rely on bringing together data in a naturally graph-like form. Combining this, via GraphRAG, with biographical details is a natural application of LLMs and could put Aficionado in a position to bring back the interactions we used to have before the days of black box recommender systems.

Built With

• docker
• llms
• postgresql
• python