03-22-2017, 10:26 PM | #1 |
FFR Player
Join Date: Apr 2009
Location: Thailand
Posts: 117
|
Dance Dance Convolution : Input raw audio - output simfiles
A recent paper named Dance Dance Convolution is out. You might be interested on reading the whole thing here if you like neural nets : https://arxiv.org/pdf/1703.06891.pdf.
But for short, neural nets is a thing which can decide based on what it had learned. (We designed the net and let it learn, but we have no idea what happens inside that it is able to deduce correct answer given a new input.) This NN will automatically detects where to put steps (analyzing "onset" in an audio file) on an unseen audio. (Shazam for example, could detect what song it is because it already learned about the song, that's not unseen) The learning material is 133 ITG files with full difficulties, 8 different note charters. Then it will decide which note to put at that point in time. The resulting simfile is in this video. https://www.youtube.com/watch?v=yUc3O237p9M Luckily it is not that intelligent to do crossovers or gallops XD
__________________
I'm actually a Sargon/5argon Last edited by tunsz555; 03-23-2017 at 09:23 PM.. |
03-22-2017, 10:34 PM | #2 |
Banned
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
This
Might be very sick In all honesty this is probably something we've all thought about but never really taken into serious consideration because of the sheer complexity in making such a program. But it's here now! |
03-22-2017, 10:40 PM | #3 |
Banned
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
From the website:
"The network that predicts step timings has no concept of rhythm or tempo. It simply answers the following question 100 times a second: should there be a step here? We map these to step charts by creating measures with 192 steps at 125BPM. We will release a script soon allowing you to manually set the tempo/offset of a chart to clean things up a little. For now, turn off colored note skins to avoid confusion." It looks like the remnants of FFR's antiquated system have washed out on the shores of San Diego. |
03-22-2017, 11:00 PM | #4 |
Confirmed Heartbreaker
Join Date: Jul 2012
Age: 35
Posts: 5,858
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
this is neat kind of reminds me of dancing monkey
__________________
|
03-22-2017, 11:08 PM | #5 |
~ お ま ん こ ~
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
my adviser sent this to me for lab research today lmao
~ S C I E N C E ~ |
03-23-2017, 02:42 AM | #6 |
FFR Player
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
Last edited by FightABunny; 03-23-2017 at 02:55 AM.. |
03-23-2017, 02:52 AM | #7 |
ur worst nitemare
Join Date: Jan 2003
Age: 32
Posts: 1,626
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
hello duderino i am taking a class on stuff like this atm and i have a couple friends who are working on something similar
beat detection is pretty decent and the patterns chosen are generally quite playable on pad (though on the song i tested i ended up with one 175 bpm 16th note minijack and a 32nd triplet)
__________________
some feathery f**k Last edited by lurker; 03-23-2017 at 02:55 AM.. |
03-23-2017, 05:17 AM | #8 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
I feel like this could be massively improved just by allowing the user to input the song's BPM if it is known, rather than try to shove the song through 125BPM which would inevitably cause nearly every arrow to be a few ms offbeat
__________________
|
03-23-2017, 07:38 AM | #9 |
FFR Player
Join Date: Dec 2007
Location: nima
Posts: 4,278
|
this is actually a fairly neat and well written paper and if i cared more about pad i'd prolly take the time to scrutinize it further
just want to point out that this paper defines an increase in difficulty as a successive increase in notes per second relating to a successive decrease in sound audibility ------ after throwing ghost love score at it a few times i am more impressed by the paper than the results though i supposed it was designed for stepmanias musics and not, you know, real music ------ ok you know what i bet i could make a reasonably playable dump file by throwing a song at it 10 times and overlapping the results Last edited by Velocity; 03-23-2017 at 09:04 AM.. |
03-23-2017, 08:08 AM | #10 |
Banned
Join Date: Sep 2016
Posts: 191
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
|
03-23-2017, 01:37 PM | #11 |
Wubalubadubdub
Join Date: May 2005
Age: 36
Posts: 1,697
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
__________________
boop |
03-23-2017, 06:13 PM | #12 | |
ur worst nitemare
Join Date: Jan 2003
Age: 32
Posts: 1,626
|
Re: Dance Dance Convolution : Input raw audio - output simfiles
Quote:
this is more a side effect of peak detection working far better on sounds with sharp attacks, as i've found that it tends to miss notes in synths quite often
__________________
some feathery f**k Last edited by lurker; 03-23-2017 at 06:20 PM.. |
|
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Thread Tools | |
Display Modes | |
|
|