[xXOpkillerXx]

Originally Posted by MinaciousGrace

questions like, given a distribution of margin of error, is it more important to have an average as close to 0 as possible?

is it more important to minimize the outliers?

can you apportion relative importance? i.e. is it more important to have roughly 80% of files within 5% but with the remaining 20 having 30%+ margins of error? or would it be preferable to have 95% of files within 7.5% and the remaining 5% within 10%? 15%?

given the option do we want an average closer to +(overrated) or -(underrated) 1%? why?

Prior to having done any modeling yet for the difficulty, my take on that is that it would initially be better to aim for a higher rate of very good guesses than minimizing outliers' error. The reason is that I could have information on what kind of files really don't fit my model. From those results I can then make more accurate tweaks to the initial model, and repeating the process until some trivial threshold is attained. Only then would I maybe sacrifice overall accuracy if the payoff is good in terms of the amount of files that are subjectively not far from expectation. Mind you, like I mentionned in earlier posts, I'm Not at the stage of implementation/tests; I cannot give you a detailed explanation of my plans because I have yet to see what primitives/attributes I can extract from the files (the reason of this thread).

Originally Posted by MinaciousGrace

how do you examine and test for this? how do you go about eliciting results specific to your goals? how do you ensure that any methods employed don't produce undesirable effects on the results? are some undesirable effects worth a closer adherence to your goal?

how much do you account for human subjectivity when testing for this? think you're going to use neural networks and match it to a score base? wrong again you just exposed yourself to population bias which, going back to the previous point, exposes you to wild outliers (30%+) of players even if it fits well with most other players

Since this can only be an unsupervised problem if we want to keep some sort of numerical range as output (which I believe we obviously do), then the results can only be trusted or not. FFR's difficulty spectrum still has flaws, but it's been worked on for a long time by expert players (OWA for example), so even though we don't want to use it as groundtruth, it's still a good indication of how accurate the predictions are (even if it's not a set quantitative measurement). The prediction accuracy is definitely harder to judge when aiming for a precise fit to subjective expectation because it's unsupervised. It then seems wiser to get a close enough fit and formulate properly what explains the variations, so that the subjective opinions can be compared to what the model predicts, and if no common grounds can be found, go back to tweaking the model and adjusting primitives.

Originally Posted by MinaciousGrace

you also have the least amount of data on the files you are most concerned with, which are the files that are the hardest and least played, because the files where there is the most player subjective agreement are the easy files that people have played to death over and over

Although this is obviously a problem than many people mention, I still have ideas to try. Depending on what model turns out to be acceptable, if any, a study on the behavior of each primitive when difficulty ramps up can potentially be extrapolated to new data. Can't make any more assumptions before having fully defined my primitives first.

Originally Posted by MinaciousGrace

how do you extrapolate existing player scorebases to new files?

I don't plan on using scores to estimate anything, but rather the existing difficulties for the ingame files.

Originally Posted by MinaciousGrace

do you apply neural networks to pattern configurations? how do you detect patterns? you already threw out the possibility of doing so, so that leaves you without that option. too bad

I won't detect patterns in a hardcoded way. I will deal with densities and various nps change distributions to accomodate for the very many ways a section can be difficult. For example, a high nps on a single receptor with fairly low nps on all 3 other receptors with minimal change can represent anything such as runningmen, anchored jumpgluts or anchored polyrhythms. The representation of patterns is still there, but not as hardly set in stone since there can be too many ways to mix patterns and very little possibility to stay more on the objective side when explaining the resulting difficulty. There's no way I can imagine someone be objectively talking about the difficulty of a runningman pattern with a minijack on every other anchored note. Patterns are friendly concepts for us to communicate about files with an easy mental visualisation, they are not a suitable difficulty metric.

Originally Posted by MinaciousGrace

even if you didn't, how do you mathematically model pattern difficulty, how do you account for subjective player strengths given extremely specific patterns and extremely specific players? do you?

I don't model patterns. Strenghts are objective and difficulty does not have anything to do with them, so no I don't account for them. If a player is good at something, then so be it.

Originally Posted by MinaciousGrace

again, the same question but applied to specific patterns, is it more important to be generally accurate and leave open high error margins on outliers or sacrifice general accuracy in an attempt to account for the outliers as best as possible? how does the decision you make impact the overall correctness?

I believe I have answered this in the above replies.

Originally Posted by MinaciousGrace

how do you deal with transitions? are transitions important? trick question, yes you fucking idiot

You never defined transitions to begin with. However, I'd say I can deal with those with nps change rate per receptor. For example, a roll to a jack will clearly show a drastic increase on one of the receptor's nps and a decrease on all other receptors. This applies to even the most bizzare patterns since nps is a distribution over time and not a finite set of patterns.

Originally Posted by MinaciousGrace

do you model stamina drain? how do you model stamina drain? physical? mental? ffr requires additional consideration for mental stamina drain because of the aaa difficulty goal. is that objectively stupid? yes, will it change? probably not

This I would say is one the the more interesting questions you've asked. Yes, FFR difficulty is judged based on AAA, so there definitely has to be a primitive for the song lenght or something similar. Average nps, mixed with the rest, can account for stamina drain but that might need some tweaking too. I do believe that using the nps change rate is helpful here also because a constant nps for a long time is more stamina draining than shorter hard sections. In the case where it's subjectively hard to tell, other primitives like max nps will hopefully lead the model to making an acceptable prediction.

Originally Posted by MinaciousGrace

the answers to these questions will guide your specific implementation, none of which you have clearly bothered asking, which is the same predictable fallacy that everyone falls into

Mentionned a few times that it's preferable to extract primitives first and then see what modeling can be done.

Originally Posted by MinaciousGrace

you aren't going to reduce file difficulty to 2 prominent variables and even if you could i don't think you would be able to use that information to actually produce a single number and assuming you did you'd still be stuck with the inherent fallacy of using machine learning to produce values that you can't actually corroborate because of human subjectivity

By "2 prominent variables" I guess you meant any decently sized quantity of variables. As for the machine learning part, that's basically the whole foundation behind unsupervised algorithms: the model gives you an output which is meant to be closely analysed to find information about your data and compare it to your subjective expectations.



Sadly (not really but w/e) you are banned, so you won't be able to reply to this soon I suppose. I would've gladly listen to your arguments as to why I'm wrong on certain points, because there is no way I can be right on all that right off the bat. Hopefully you learn to have a respectful conversation/debate before you're unbanned though.