Music Technology

An A.I. attempts to rewrite Thomas Tallis

Jukebox is a type of neural net – an network of artificial nodes which is ‘trained’ on a series of data, and can then be taught to use this data to generate new strings. These artificial intelligence networks have been used to create unique images, poetry, scripts, and music. Essentially, they work from one data-point to the next and try to work out what letter, pixel, or note should come next, based on its training. I first encountered them on the wonderful AI Weirdness blog, which is a rabbit-hole of the hilarious and surreal things that can now be done with this technology.

What makes Jukebox different from many of the varieties of generative music that have come before is that it’s trained not on symbolic datasets – for example MIDI files which encode digital musical instructions into code – but actual audio. Not only that, but it has also been conditioned to recognise the shape of words, meaning it can – sort of – generate these sounds too.

This means that you can feed it an audio sample, give it a few parameters such as a genre or artist to emulate, specify the words, and then ask it to predict what should come next. It bases these choices on what it has learned about the 1.2 million real songs that formed its ‘training’ dataset.

The results, as one might expect, vary wildly in quality. On the aforementioned blog, Janelle Shane posts some creations which are exciting and not a little horrifying – for example, a pastiche Frank Sinatra Christmas song which should belong to an album entitled ‘Music from the Uncanny Valley’.

Most of the results that have so far been posted by researchers have the flavour of I’m Sorry I Haven’t A Clue’s ‘One Song to the Tune of Another’ (see here if you need a description of this very complicated game). Thus you can get the AI to do Queen in the style of Nirvana, for example.

Inevitably, a large majority of its training data is non-classical in nature, but I still thought it would be interesting to prompt it with some choral music, to see what it would come up with. The results are surprisingly impressive, though naturally very odd.

Jukebox was primed with about twelve seconds of a recording of the classic Thomas Tallis banger ‘If ye love me’, and given the full lyrics. Now, it has a limited dataset of genres and artists to use as a template, and the closest I could find were ‘Classical’ for the genre and, yes, ‘Mormon Tabernacle Choir’ for the artist. Already the mind boggles.

It had three goes at generating 40 more seconds of the piece, transforming the input through a process of ‘upsampling’ at three different levels. Let’s have a listen to what it came up with after some four hours of labour:

1. If ye love meh

The neural net takes over on the last syllable of ‘commandments’, and in each sample it has a different idea of what chord should follow. Here, it plays it safe and repeats the chord, which works. It’s cool that it makes the phrase lengths broadly ‘vocal’ in nature, and simulates breaths before them too, presumably learning to ape the opening of the prompt.

Some extraneous, non-vocal sounds start to appear in the middle, including at one point what sounds like a train passing, or perhaps a snare drum. I wonder if that’s due to it using the Mormon Tabernacle Choir, with their often quite elaborate arrangements, as a model. For all it knows, the piece starts acappella and then goes on to become instrumental. It could also be misinterpreting the acoustical reverb as ‘new sounds’ in their own right, and trying to work out what they could mean.

It also mostly stays in key, until the very end, which normally unremarkable thing I point out as it is not a given in the other samples…

2. If ye…love…meeee….

This one’s ‘-ment’ chord is actually a cool choice – A minor rather than original F major. Afterwards, however, it goes off the rails a little earlier than the previous one. I like the little cymbal ‘ting’ after the second phrase. The choir’s vocal production becomes very slurred, and the AI forgets the key, if it ever knew what that was in the first place. The end becomes rather worrying and distorted, and the harmony is bizarre.

Presumably, because it isn’t given any information about what harmony actually is, it doesn’t know the rules except by what it’s heard before. It must base its moment-to-moment choices about what audio to generate on what previous bits of audio it knows are usually followed by. However, I can’t imagine there are very many examples in the dataset of an audio progression of the sort that happens at the end of this excerpt. How did Jukebox come up with it?

3. If ye love me, keep in the same key..?

Uh. Pretty out-there choice of a continuation chord on ‘commandments’, but it recovers pretty successfully and sticks the landing. The words also feel a little more present in this one, and it stays in a key and sort of in tune longer than the others, at least until a demonic final entry before the file mercifully ends. There’s some intriguing parallelism in the middle, during the extension of a word that I think might be ‘you’. And it remembers to be acappella throughout, which the other two didn’t manage. Probably the most successful.

What’s impressive is that, in all three of its goes, the AI learns that the phrases are preceded by breaths, and apes the length of the first phrase for most of the following ones, varying them subtly but plausibly. But the overall effect of the continuations (if one can ignore the ghostly distorting of the voices) is of someone dreaming a conclusion to a piece to which they only remember the opening. Like dreams, they lose coherence and stop making sense at various points. Still, given that the vast majority of its training is on popular music and other styles, it does a pretty creditable if slightly meandering job.

For me, the results of this are roughly equal parts disturbing, exciting, and hilarious. Disturbing, because the distorted voices end up sounding like something from a horror film. Exciting, because the computer isn’t bound by our conception of harmony or structure – it dreams up new combinations that we might never have thought of. Insomuch as it has worked out the rules, it’s done so by simply listening to a lot of music, like an alien tuning in from another planet and trying to understand how our music works.

As a tool for inspiring creativity, it has limitless potential, because it can always surprise us with its choices. It won’t be long before it gets better at understanding different genres and is able to produce highly competent pastiches – the musical equivalent of these non-existent people.

In the meantime it’s more likely to make me giggle than reflect on the mysteries of human existence. But it won’t be long. I, for one, welcome our new robotic musical overlords.

Choirs Conducting

Thematic concert programmes: worth the hassle?

Conductors…do not always know how to shape a meaningful whole out of smaller pieces…We often program according to some vague theme or chronological order, perhaps without real thought to or justification for our choices.

I feel, as they say, seen. The themed programme is a staple of choral concerts the world over, and yet it can often feel unsatisfying. So convinced are we of the need to theme an evening’s musical offering, to weave it tightly together to make a cogent whole, that we can often end up in an uncomfortable straitjacket. I find myself casting around for something that hasn’t already ‘been done’ in order to justify a selection of music. But is it really necessary, and can we avoid the hassle that the themed programme so often entails?

Why we use themes

The obsession with themes tends to manifest in classical choirs, and rather less so in orchestras. Partly that’s because orchestras deal on the whole in much larger chunks of music. The standard orchestral concert programme requires an overture, a concerto, and a symphony – three items, increasing in length, and usually filling up a couple of hours quite neatly. There’s often simply no need for any kind of external bracket to unify the music. Job done.

Choirs, on the other hand, have additional considerations, at least when performing on their own, or with a single accompanying instrument. The most obvious is the endurance level of the singers, reckoned generally to be lower than that of most orchestral instruments. The other is the available corpus of music, ranging from miniatures to epics, but often on the shorter side, especially where sacred music is concerned.

To compensate for the lumpy proportions of the music, choral programmes have embraced the extra-musical linking device of the theme. We want audiences to feel that what they’re hearing is a cogent hour or so’s music, and that it hangs together with some kind of consistency.

It comes from a hyper-awareness of an audience, wanting to provide them with a guide, a narrative thread, that will give them a route in to understanding and appreciating the music that the choir has prepared.

Advantages of the themed programme

A theme offers this curated experience, taking the listener lightly by the hand and leading them on a tour of whatever it is that’s being explored. A theme, whether loose or tightly-concentrated, provides a prism through which to view the music, a way to help understand and contextualise it.

Additionally, juxtaposition of items is a powerful tool to illuminate connections in all sorts of ways. Sometimes the most seemingly unlikely of segues can yield great insights into compositional process or musical sentiment. There was a recent, thrilling example of this in one of the BBC Proms’ eerily audience-less concerts this summer – Simon Rattle led the LSO straight from a Gabrieli canzona into Elgar’s Introduction and Allegro, a striking and unpredictable segue which played up the dramatic contrasts of groups of instruments in both pieces.

Of course, juxtaposition of this sort need not be confined to the themed programme per se. But another reason we value a unifying extra-musical element is that it provides an entry point, especially to non-specialists or those less familiar with the genre of music on offer. Even the presence of just one or two words, nominally applying to all the pieces in a concert, allows someone with little experience of listening to a particular kind of music to find ways to apply these descriptors, and find a way in.

Problems with themes

However, it’s very easy to get bogged down in a theme, or for these themes to become tired and lazy through unthinking repetition. It has become a standing joke in choral circles (if not a particularly hilarious one) that a newly-formed chamber choir will specialise in early music, new music, and Parry’s Songs of Farewell.

When inspiration doesn’t strike, it can be all to easy to pick a well-worn trope and use it as the basis for a programme – or to try and squeeze pieces into a theme that don’t belong there. Audiences can be forgiving (especially if you do this with a wink!), but it’s awkward when a piece is shoehorned into a programme where it doesn’t belong. Constructing a programme within a restrictive theme can be like playing Tetris in four dimensions.

Equally, in the desire to present a theme which hasn’t been ‘done’, it can quickly get quite abstruse. I remember thinking myself very clever for a segment of a Christmas concert which I entitled ‘The Three Kings’ and populated with music by Caspar Othmayr, Melchior Hoffman, and Balthasar Resinarius. I might have been royally pleased with myself – but did it add anything apart from being a little glib?

I sometimes think the use of themes in this heavy-handed way betrays a rather patrician lack of trust in the audience. Do we really feel that audiences can’t handle a programme which is simply a selection of music we want to perform? The enthusiasm gained on our part is surely much more valuable to the success of the concert.

In these situations, why don’t we simply free ourselves from the strictures of the theme – let ourselves off the hook a little? After all, audiences are rarely thinking about the intricacies of a programme when listening to it as much as we are when assembling it. Better to embrace this sometimes and simply say: here is some music that we think represents us right now. We hope you enjoy it.

There’s a palpable sense of relief in casting aside that unworkable theme and replacing it with the answer to the question: what do the ensemble and I actually want to perform at the moment? The results of investigating that question could be of much more value than a too-clever theme.

Teach without being didactic

Ultimately, I like to leave an artistic event, be it a concert or a visit to a gallery, feeling cleverer – but not simply because I’ve been taught something, but because I’ve figured it out for myself. I think the most successful programmes are ones that gently lead an audience to work something out on its own – figure out a connection, understand a form.

That’s one of the reasons the much-maligned chronological programme remains useful. Art galleries still generally arrange works in chronological order from early to late, and that works for us – we notice the developments in style and form, even if we don’t have specialised training. The clever curator leaves clues so that we can teach ourselves what they want us to learn from the exhibition.

In live performance, we know that even the most carefully-designed programme only comes to life if it is presented engagingly. When I first started programming and conducting concerts, I was very determined that the music should speak for itself. I remained resolutely tight-lipped as my meticulously planned programme segued imperceptibly from one piece to another. I’m sure there are times when this approach can work, but now I think it can alienate as much as it can draw in, especially with a new listener. These days – depending on the programme and the place and all sorts of other factors – I’m generally much more comfortable interrupting the musical flow at intervals to speak to the audience and offer a few thoughts on what to listen for, or how.

Clearly, these extra-musical elements are important, especially to those new to the form. It’s interesting that some of the most successful and artistically interesting choral presentations to come out of the Year of Hell that is 2020 have involved a heavy dose of narrative, implied or actual: Marian Consort’s sequence of collaborative filmed projects, or Stile Antico’s recent Journey of the Mayflower.

The challenge, then, is to find extra-musical narratives, be they thematic or otherwise, which help us generate programmes that we are actually excited about performing, and that audiences will find energising and informative.