Shimon, the robotic maestro from Georgia Tech’s Center for Music Technology, is releasing an album and going on tour. To write lyrics, the robot employs deep learning combined with semantic knowledge and rhyme and rhythm. Shimon has also had a complete facelift giving it a much more expressive mug for singing. In IEEE Spectrum, Evan Ackerman interviewed Shimon's creators, professor Gil Weinberg and PhD student Richard Savery:
IEEE Spectrum: What makes Shimon’s music fundamentally different from music that could have been written by a human?
Richard Savery: Shimon’s musical knowledge is drawn from training on huge datasets of lyrics, around 20,000 prog rock songs and another 20,000 jazz songs. With this level of data Shimon is able to draw on far more sources of inspiration than than a human would ever be able to. At a fundamental level Shimon is able to take in huge amounts of new material very rapidly, so within a day it can change from focusing on jazz lyrics, to hip hop to prog rock, or a hybrid combination of them all.
How much human adjustment is involved in developing coherent melodies and lyrics with Shimon?
Savery: Just like working with a human collaborator, there’s many different ways Shimon can interact. Shimon can perform a range of musical tasks from composing a full song by itself or just playing a part composed by a human. For the new album we focused on human-robot collaboration so every song has some elements that were created by a human and some by Shimon. More than human adjustment from Shimon’s generation we try and have a musical dialogue where we get inspired and build on Shimon’s creation. Like any band, each of us has our own strengths and weaknesses, in our case no one else writes lyrics, so it was natural for Shimon to take responsibility for the lyrics. As a lyricist there’s a few ways Shimon can work, firstly Shimon can be given some keywords or ideas, like “earth” and “humanity” and then generate a full song of lyrics around those words. In addition to keywords Shimon can also take a musical and write lyrics that fit over that melody.
"Musical Robot Learns to Sing, Has Album Dropping on Spotify" (IEEE Spectrum)
More: "Shimon: Now a Singing, Songwriting Robot" (Georgia Tech)
