DeepMind’s AI Learns The Piano From The Masters of The Past

Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. Today, we will listen to a new AI from DeepMind
that is capable of creating beautiful piano music. Because there are many other algorithms that
do that, to put things into perspective, let’s talk about the two key differentiating factors
that set this method apart from previously existing techniques. One, music is typically learned from high-level
representations, such as the score or MIDI data. This is a precise representation of what needs
to be played, but they don’t tell us how to play them. These small nuances are what makes the music
come alive, and this is exactly what is missing from most of the synthesis techniques. This new method is able to learn these structures
and generates not midi signals but raw audio waveforms. And two, it is better at retaining stylistic
consistency. Most previous techniques create music that
is consistent on a shorter time-scale, but do not take into consideration what was played
30 seconds ago, and therefore they lack the high-level structure that is the hallmark
of quality songwriting. However, this new method shows stylistic consistency
over longer time periods. Let’s give it a quick listen and talk about
the architecture of this learning algorithm after that. While we listen, I’ll show you the composers
it has learned from to produce this. I have never heard any AI-generated music
before with such articulation and the harmonies are also absolutely amazing. Truly stunning results. It uses an architecture that goes by the name
autoregressive discrete autoencoder. This contains an encoder module that takes
a raw audio waveform and compresses it down into an internal representation, where the
decoder part is responsible for reconstructing the raw audio from this internal representation. Both of them are neural networks. The autoregressive part means that the algorithm
looks at previous time steps in the learned audio signals when producing new notes, and
is implemented in the encoder module. Essentially, this is what gives the algorithm
longer-term memory to remember what it played earlier. As you have seen the dataset the algorithm
learned from as the music was playing, I am also really curious how we can exert artistic
control over the output by changing the dataset. Essentially, you can likely change what the
student learns by changing the textbooks used to teach them. For now, let’s marvel at one more sound sample. This is already incredible, and I can only
imagine what we will be able to do not ten years from now, just a year from now. Thanks for watching and for your generous
support, and I’ll see you next time!