Dance Dance Convolution : Input raw audio - output simfiles

[tunsz555]

A recent paper named Dance Dance Convolution is out. You might be interested on reading the whole thing here if you like neural nets : https://arxiv.org/pdf/1703.06891.pdf.



But for short, neural nets is a thing which can decide based on what it had learned. (We designed the net and let it learn, but we have no idea what happens inside that it is able to deduce correct answer given a new input.)

This NN will automatically detects where to put steps (analyzing "onset" in an audio file) on an unseen audio. (Shazam for example, could detect what song it is because it already learned about the song, that's not unseen)

The learning material is 133 ITG files with full difficulties, 8 different note charters.

Then it will decide which note to put at that point in time. The resulting simfile is in this video.
https://www.youtube.com/watch?v=yUc3O237p9M

Luckily it is not that intelligent to do crossovers or gallops XD