Auto chart generator achieves 60% predicting accuracy on existing challenging+ charts

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

Are you one of the contributors? Wheres 60% from and how does one do suectivity accurately, I'm curious of a little more before I go clicking around more, such as where you got that info. Im no neural network expert but 60% is only 10% more than 50 at which Id assume its still random 50/50, with a sample size that happened to be 60% "accurate", where other sets could be 30, 40, 80, random is random

Edit: tl;dr more info?

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

No, I'm not a contributor, but I happens to work on very similar topics. And well yeah, it depends on how you treat that 60% accuracy. This is just a preview and we may have to wait for a few months before all details are available.

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

Hmm now I need to figure out what to do with a .tex, I've been interested in convolutional and non neural networks but havent had a topic of interest to dive in, I'll be watching this thanks.

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

there's already a thread about this here
but the primary thing to take into account here is that this is specifically about pad play (ddr/itg/etc) and definitely not ffr

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

ie, not relevant and never going to have any idea of charting techniques that actually require a level of skill on keyboard to play.

it seems like a time-sink in all honesty, they should do something more constructive with their skills rather than designing something thats never going to reach full potential

like actually just make the charts themselves

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

The main take-away here, as stated in the paper, isn't so much that "hey guys this is actually going to help DDR developers/artists", but rather "hello fellow scientists, here is a huge fucking corpus of annotated data, use this for your research purposes in designing intelligent music information retrieval systems"


there's no reason at all why this can't be applied to keyboard charts

the corpus they used was from ITG, so the techniques the LSTMs learned were about ITG. This research would theoretically be extensible to keyboard charts if they had a large enough corpus

and in the time it would take a human to produce a chart, the LSTM/CNN method would produce thousands of charts, if it had enough songs to work with

the tradeoff is that most of those charts would suck major ass, but this is far from useless tbh

EDIT: I don't really make FFR charts, and I'm assuming that there is no fundamental difference between charting for these different games, a difference that stops neural networks from learning patterns altogether

if there is, placing my foot in my mouth

There are two steps that are explored here:

1) Given a song, where do we place the notes? (on this quarter notes, eighth note, etc)
2) Given a spot in the song to place notes, what notes do we place there? (Left, right, up, down, etc)

For the first one, you can expect a baseline of 50%: either one time-step in the song gets a note, or it doesn't. The 60% might not seem that impressive, but it IS better than the baseline, which is still pretty good.

For the second one, you can expect a baseline of 100/(C(4, 4) + C(4, 3) + C(4, 2) + C(4, 1)) = 100/15 = 6.66% per placement. (After all, each placement can have Left, Right, Up, Down, Left and Up, Left and Down, etc)

I might be misinterpreting something here because tired, so somebody can correct me if I'm wrong, but 50% for this part is pretty damn good.

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

Oh cool thanks.

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

Personally I want it to make really shitty minijack infested charts, they would be terrible but it would be fun to play

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

there are pretty large differences in making charts for pad and kb play
1 - pad patterning is somewhat restricted due to the physical aspect of the playstyle, and kb patterning is not
2 - patterning approaches are extremely varied in kb play and a fully functional tool would not only need to be technically accurate, but would also need to be artistic as well

I'm sure someone else can expand on these points further, I'm busy atm

this is a really neat idea but it would probably only ever work for 4k pad play.

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

It's going to be difficult to respond to this because I think there are some irreconcilable differences in opinions here, but I'm going to try anyway because I want to clarify some misconceptions

1) Research is iterative, and results like this do not mean that this is ready to replace human beings (spoiler alert, it will never be ready to replace human beings). What is promising here is that with this preliminary research shows strong results. And yes, these are strong results.

2) The paper clearly indicated that the method was actually better at predicting higher difficulty charts, which is at least in part a consequence of higher volume of steps.

3) "It's never gonna happen, machines lack imagination dude"
This really pisses me off, tbh, because it's so closed-minded. It's a common human arrogance that "humans are the only creative things out there." Can you define what creativity is? What imagination is? I strongly doubt that you can conjure a definition of either of those terms that is exclusively human-centric, because cognitive scientists and AI researchers have tried for years to classify, operationally, why humans are in monopoly of either of these things. This isn't really central to your argument, but I want to make clear that this particular statement is not as "obvious" or "trivial" as you are making it seem.

4) "But I'll take your word that 50%'s "pretty damn good." you've played these games enough to know that fucking up 1 in every 2 notes is no big deal."

The "50%" here is the ability of the system predict one particular step artist's charts for one particular song. I can see where you're coming from here, the ability to emulate the timing of one particular human being 50% of the time is not great for your purposes right now, but it is incredibly short-sighted of you to say that "it's never gonna happen", which is an expression you used multiple times. Refer to point 1.

5) @everyone who asked, I realize there are different stepping patterns between different games. What I'm asking is whether there is a particular reason why it is not possible to teach somebody how to step for keyboard games.

Presumably, there are a set of "rules" and "heuristics" for good tracks. People talk about good and bad tracks all the time. People here have some understanding of "when" and "where" to place steps. People can learn how to make good tracks. Ergo, there is a logic, or a set of logics, that human authors rely on when making tracks. This is what the system is learning. This is not different from DDR/ITG. It makes no fundamental difference that keyboard is more complex. (Until we get results for such an experiment, that is.)

It's like training for a 10k vs training for a marathon. The training process is not fundamentally different. The latter is harder and requires more resources and time, but you are still running.

It is not the case that this is the difference between training for a marathon and training for a hot-dog eating contest.

Anyway, sorry for the emotional response, but I feel an obligation to set the record straight about things

-- an AI PhD student

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

DaBackpack, I think the short-sighted comments result from people not seeing the long-term potential for this project. Machine Learning and AI are rapidly growing fields and this project is an example of a program that recognizes patterns based on what it detects.

I can imagine this project being posted in an ITG players group and receiving judgmental responses about the program making "poorly patterned" or "bad" charts because that's the general attitude of that community -- if you don't do something a certain way, it's unplayable and sucks.

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

I think this is actually the better analogy.

Being a god at ITG doesn't make you proficient at FFR/SM, and vice-versa.

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

This analogy was meant for the chart authoring process, not playing

a better analogy would be

"Writing in English and writing in French are different tasks, but the task -writing- are the same."

"Writing in English and shitting on the carpet are different tasks, and are not related."

Re: Auto chart generator achieves 60% predicting accuracy on existing challenging+ ch

Ok. Being a god at charting for SM/FFR doesn't make you proficient at charting for ITG and vice-versa.