Flash Flash Revolution: Community Forums Scoring Data in Brutal Difficulty Range
 Register FAQ Members List Calendar Search Today's Posts Mark Forums Read

 08-26-2019, 11:16 PM #21 xXOpkillerXx ✘ Forever OP✘     Join Date: Dec 2008 Location: Canada,Quebec Posts: 3,900 Re: Scoring Data in Brutal Difficulty Range Your idea of variable decay is basically something I brought up in chitchat discord but less extreme; the ideal formula would be one that varies for each chart. Let f(c, s) = a Where c is a chart, s is the raw goods score and a is the resulting AAA equivalency. This gives much flexibility over various chart structures and can be implemented without too much trouble. Of course, that requires some computed difficulty, hence my work on that some time ago. I'll see if I can extract some useful stats for this thread this week. I dont think the current system is "fixable" without big changes, it simply does not account for chart structure, which it should.
08-26-2019, 11:27 PM   #22
FFR Veteran
Skill Rating Designer

Join Date: Jan 2016
Age: 24
Posts: 269
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by xXOpkillerXx I dont think the current system is "fixable" without big changes, it simply does not account for chart structure, which it should.
Well, the current system relies entirely on the manually-assigned difficulty rating to sum up all the factors like chart structure. To do anything different, we'd basically need an automatic difficulty calculator like Etterna's.
__________________

Zenith Ultimate Struggle: 3rd (D2)

All public 1-36s AAA'd

08-26-2019, 11:37 PM   #23
xXOpkillerXx
Forever OP

Join Date: Dec 2008
Posts: 3,900
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by RenegadeLucien Well, the current system relies entirely on the manually-assigned difficulty rating to sum up all the factors like chart structure. To do anything different, we'd basically need an automatic difficulty calculator like Etterna's.
Well, yes.

Although Rob's idea would be a patch for the apparent edge cases, at some point if we want accurate difficulty we need better maths, that's just how it is. I got so much shit for trying to make a calc, yet it's clearly the way to go, especially for a game like this with so many charts.

Edit:
The manual difficulty can only be the AAA difficulty. The AAA equivalency formula computes a score for any raw goods count. It is evident that the mapping makes 0 sense.

Last edited by xXOpkillerXx; 08-26-2019 at 11:42 PM..

08-26-2019, 11:39 PM   #24
One Winged Angel
Anime Avatars ( ◜◡＾)っ✂╰⋃╯

Join Date: Mar 2007
Location: Squat Rack
Age: 30
Posts: 10,761
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by RenegadeLucien Well, the current system relies entirely on the manually-assigned difficulty rating to sum up all the factors like chart structure. To do anything different, we'd basically need an automatic difficulty calculator like Etterna's.
The current system needed more information to work anywhere near accurately to begin with. Charts have a single number attached to them to approximate the value of a near perfect or perfect score, but the growth rates approaching that point can be widely different depending on chart structure and skills tested, and this had been publically acknowledged. The current system erroneously slapped on identical decay formulas to every chart within a given subtier using a few inputs and comparisons from surveying the event team staff and making modifications as necessary until it looked 'nice'. Much more work needed to be done to capture accurate equivalencies and I voiced that prior to the system's release but no one seemed to care until the problem became much more evident several years later.

Scores on RATO/DP from D6/D7 players were spitting out equivalencies in the FMO or lower range for years but no one paid any mind because it was just a couple charts and sure whatever that's fine I guess. But now that there's gonna be more and an entire division is going to be reliant on most of that range for what comprises their skill rating, that definitely needs to change.
__________________

Quote:
 Originally Posted by ilikexd i want to be cucked by cirno

Last edited by One Winged Angel; 08-26-2019 at 11:43 PM..

08-26-2019, 11:54 PM   #25
FFR Veteran
Skill Rating Designer

Join Date: Jan 2016
Age: 24
Posts: 269
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by xXOpkillerXx I got so much shit for trying to make a calc.
Who gave you shit aside from Mina? I don't think anyone here would actively oppose you or anyone else trying to make a difficulty calc.

Quote:
 Originally Posted by One Winged Angel Scores on RATO/DP from D6/D7 players were spitting out equivalencies in the FMO or lower range for years but no one paid any mind because it was just a couple charts and sure whatever that's fine I guess.
It's been stated over and over and over again by so many people that the current system's accuracy drops very fast as you go higher up in good count; I don't think this was ever an issue unique to DP/RATO or even high difficulty songs in general.
__________________

Zenith Ultimate Struggle: 3rd (D2)

All public 1-36s AAA'd

08-27-2019, 12:15 AM   #26
One Winged Angel
Anime Avatars ( ◜◡＾)っ✂╰⋃╯

Join Date: Mar 2007
Location: Squat Rack
Age: 30
Posts: 10,761
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by RenegadeLucien It's been stated over and over and over again by so many people that the current system's accuracy drops very fast as you go higher up in good count; I don't think this was ever an issue unique to DP/RATO or even high difficulty songs in general.
And yet the system remains the same. I'm aware it's evident elsewhere, it was just most glaring on those charts.

I don't see an issue with trying to hammer out any and all issues in an effort to create a more accurate system. I feel like you take these comments as personal attacks and are quick to displace blame elsewhere, such as on the difficulties having needed to account for this when this was a system assumptive of numerous chart qualities being meticulously considered and represented by a single number so as to treat them identically when extrapolating a score's worth.
__________________

Quote:
 Originally Posted by ilikexd i want to be cucked by cirno

 08-27-2019, 12:55 AM #27 RenegadeLucien FFR Veteran Skill Rating Designer     Join Date: Jan 2016 Age: 24 Posts: 269 Re: Scoring Data in Brutal Difficulty Range I'm not sure where I gave off the impression that I'm taking any of this personally, but if that's what you're getting, that wasn't my intention, I'm sorry for giving off that impression. The current system isn't even mine. I'm pretty sure we're on the same side here. The system needs to change. My point is that the only way we're really going to get something that's truly accurate is with a calculator. Yeah, we could have some sort of variable decay rating for each song that tunes the base skill rating formula for it, but to do that we'd need a calculator anyway, unless someone wants to manually go through all 2000+ songs and give all of them another number. __________________ Zenith Ultimate Struggle: 3rd (D2) All public 1-36s AAA'd
08-27-2019, 08:10 AM   #28
xXOpkillerXx
Forever OP

Join Date: Dec 2008
Posts: 3,900
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by RenegadeLucien Who gave you shit aside from Mina? I don't think anyone here would actively oppose you or anyone else trying to make a difficulty calc.
Mina was the most direct about it but many in discord would keep saying that I'm wasting my time and be pretty passive-aggressive. Mostly people just bandwaggoning with Mina. I admit that I have a problem with people who say something is not possible without being able to provide a thorough proof of their claim. Many just spew some pseudo maths arguments but cant get into details.

Anyway, if interest for a calc goes up for real now, I might have some motivation to help.

Last edited by xXOpkillerXx; 08-27-2019 at 08:24 AM..

 08-27-2019, 08:54 AM #29 xXOpkillerXx ✘ Forever OP✘     Join Date: Dec 2008 Location: Canada,Quebec Posts: 3,900 Re: Scoring Data in Brutal Difficulty Range Here's my take on the difficulty factors of individual notes: -Local one-hand complexity: how difficult it is to hit that note given the past and future X notes on the same hand (future notes are necessary to account for readability). That is something I havent finalized, but generally the difficulty goes up first with the spacing of notes (the less frames between the notes, the harder it is in a non-linear fashion so that 1-framers are much harder than 2-framers but 30 and 31 frames are pretty similar), then by transition (at the same speed, a jump to a single note is always harder than a minijack or a jumpjack, and single-to-single like 12 or 34 or 21 or 43 have a special weight for being easily hit as a jump or not). Some time-based gaussian window over each note gave decent results with a window of about 1 second (30 frames) or less on each side and a low std dev (< 1.0). -Global 2-hands complexity: a distribution of the two 1-hand complexities at each timestep. For example, a very hard section on one hand with a very simple one on the other hand could be easier than medium difficulty on both hands at same time. This would need refinement for polys and a more well-defined explanation (with general and edge cases). -Note time: just a factor of where the note is in time. This has to be picked/formulated so that a note after 5 minutes with low complexity cannot be harder than a note 1 minute in with high complexity. It is easily defined once the complexities are defined. Accounts for focus loss and partly for stamina. -Note stamina: a large time-based past-only window over the aggregated factors above. Essentially accounts for breaks in a song; it's easier to hit a hard section after a break than in the middle of some stream or whatever. This gives a difficulty number to each note of a file. It then becomes possible to compute a different AAA equiv formula for each file. The overall difficulty of a file would then not be a single number, but rather a distribution over raw goods count. Nothing forbids us to compute and show difficulty for a specific count (AAA difficulty, 10g difficulty, 20g difficulty, etc). I have a pretty good setup to compute these already, so I'm saying it here to gather more opinions on the aggregation part and various factors (gaussians parameters, complexity, etc). @rob if you prefer this to be in another thread let me know Last edited by xXOpkillerXx; 08-27-2019 at 09:10 AM..
 08-27-2019, 11:37 AM #30 Dynam0 The Dominator     Join Date: Sep 2005 Location: North Bay, ON Age: 31 Posts: 8,839 Re: Scoring Data in Brutal Difficulty Range My main concern with that approach is pattern manipulation and stamina. Getting those right would be a tough ask imo. I still think it is far less tedious to have subjective difficulty assignments and as Rob said we just need to get the decay part correct. It's not incredibly far off at this point.
08-27-2019, 11:42 AM   #31
xXOpkillerXx
Forever OP

Join Date: Dec 2008
Posts: 3,900
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by Dynam0 My main concern with that approach is pattern manipulation and stamina. Getting those right would be a tough ask imo. I still think it is far less tedious to have subjective difficulty assignments and as Rob said we just need to get the decay part correct. It's not incredibly far off at this point.
Can you give me specific examples ? Either from actual files or just made up sections ? It's hard to get into details without concrete examples.

PS: I already have computed the manipulatable sections mostly. That's dealt with by accounting for the number of frames you have to hit singles as jumps (1 frame being harder to manip, 2 frames is most likely a jump). As for stamina, I explained a basic framework for it; what would you disagree with ?

Edit: I also disagree with "it's not incredibly far off"; far off what ? The only thing the current system can hope to reach is optimal subjective consensus (as in the most people who agree with the difficulties). While that isn't a bad metric per se, it is inevitably flawed and biased, and doesnt solve the problem: variable decay is still just a decay, it doesnt fully compensate for chart structure, it just covers cases where the distribution of ordered note difficulty ressembles one unique function with a modifyable decay. The truth is there can be many more shapes to that distribution.

Last edited by xXOpkillerXx; 08-27-2019 at 12:03 PM..

 08-27-2019, 12:28 PM #32 Dynam0 The Dominator     Join Date: Sep 2005 Location: North Bay, ON Age: 31 Posts: 8,839 Re: Scoring Data in Brutal Difficulty Range Idk man I just think taking this granulated an approach to difficulty is not worth the effort and it's always going to be prone to outliers reliant on fudge-factoring. Who determines how difficult a trill is a 240bpm? Someone who is good at them? The average player? Who is an average player? You see how silly the concept of automating this is? It's entirely subjective. There will still be 97s that feel like 96s and so on based on individual player strengths.
08-27-2019, 12:40 PM   #33
xXOpkillerXx
Forever OP

Join Date: Dec 2008
Posts: 3,900
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by Dynam0 Idk man I just think taking this granulated an approach to difficulty is not worth the effort and it's always going to be prone to outliers reliant on fudge-factoring. Who determines how difficult a trill is a 240bpm? Someone who is good at them? The average player? Who is an average player? You see how silly the concept of automating this is? It's entirely subjective. There will still be 97s that feel like 96s and so on based on individual player strengths.
That pattern approach never worked for me. The questions you're asking are legit ones, but that's like surface level. I'd really appreciate to debate more on the points I mentionned but you have to be willing to talk details, otherwise this discussion goes nowhere (which is often the case). I'm not saying a calculator will achieve 100% agreement from everybody on every difficulty, but it would give a much better framework to argue on.

 08-27-2019, 12:45 PM #34 Dynam0 The Dominator     Join Date: Sep 2005 Location: North Bay, ON Age: 31 Posts: 8,839 Re: Scoring Data in Brutal Difficulty Range Well considering my prior post stated that going into said details is not worth the effort, I am out of this discussion :P Best of luck though, if you do figure out a way to do it then I'll be proven wrong and then some
 08-27-2019, 03:29 PM #35 SputnikOwns The Frog     Join Date: Sep 2007 Posts: 160 Re: Scoring Data in Brutal Difficulty Range Correct me if I'm wrong, but isn't it quite rare to have an AAA equal to or above one's rating? The algorithm is excellent as far as I'm concerned -- so long as the song difficulties are correct. __________________ Last edited by SputnikOwns; 08-27-2019 at 03:30 PM..
08-27-2019, 03:44 PM   #36
Matthia

Join Date: Nov 2017
Location: Pacific Timezone, USA Age: 18.8
Posts: 314
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by SputnikOwns Correct me if I'm wrong, but isn't it quite rare to have an AAA equal to or above one's rating? The algorithm is excellent as far as I'm concerned -- so long as the song difficulties are correct.

shit like this happens but only because of how absurdly low the difficulty of this file outside of two or three main hard parts is compared to what it is at currently

Edit: It is very likely that White Walls Part 2 is also seeking a nerf not much in AAA difficulty but rather the How-Easy-This-Can-Be-Abused factor which is the overall discussion we are having at the moment

Last edited by Matthia; 08-27-2019 at 03:48 PM..

08-27-2019, 04:07 PM   #37
xXOpkillerXx
Forever OP

Join Date: Dec 2008
Posts: 3,900
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by SputnikOwns Correct me if I'm wrong, but isn't it quite rare to have an AAA equal to or above one's rating? The algorithm is excellent as far as I'm concerned -- so long as the song difficulties are correct.
The thing is they're not, which is why this thread exists

 08-27-2019, 04:40 PM #38 SputnikOwns The Frog     Join Date: Sep 2007 Posts: 160 Re: Scoring Data in Brutal Difficulty Range That's totally fine of course. They should be ordered correctly according to top players. No need to adjust the algorithm for 100+ though. __________________
08-27-2019, 04:45 PM   #39
xXOpkillerXx
Forever OP

Join Date: Dec 2008
Posts: 3,900
Re: Scoring Data in Brutal Difficulty Range

Quote:
 Originally Posted by SputnikOwns That's totally fine of course. They should be ordered correctly according to top players. No need to adjust the algorithm for 100+ though.
This is the second problem. As it was said a few times, there are a bunch of files which are hard AAAs but easy SDGs, and that breaks the current system.

08-28-2019, 05:40 PM   #40
One Winged Angel
Anime Avatars ( ◜◡＾)っ✂╰⋃╯

Join Date: Mar 2007
Location: Squat Rack
Age: 30
Posts: 10,761
Re: Scoring Data in Brutal Difficulty Range

I agree with Dynamo, honestly there are far too many variables to deal with to ensure a calculator will work effectively. Manually establishing separate decay formulas for charts that are more accessible to less skilled players will reach higher levels of accuracy and communal consensus without the hassle of dealing with all the intricacies needed to be taken into account for a calculator in a game that encodes charts at 30fps. A 16th roll at 300bpm starting on a quarter note can effectively be jumptrilled because the one frame gaps all coincide between arrows hit with the same hand (1/2 and 3/4), but offset the roll by a 16th and now all the one frame gaps appear at 2/3 and 4/1. Having to account for quite literally the same pattern being variable in difficulty depending on the starting note, on top of everything else that needs to be taken into consideration...I personally wouldn't want to invest any time in that.

What Dynamo was fudging around with in Excel looks interesting and I might mess around with that later.

Tangential to that, I've looked at suggestions and scoreboard data and am interested in feedback for this structure of 98+:

A few notes:

* denotes charts that would benefit from separate equivalency decays. I'm not suggesting they should share identical formulas, but a manipulation of the current formula is required.

La Camp's scoring data after a decade in game suggests it can hang with the 99s. Structurally I may not entirely agree with that shift but it fills out the 99s a bit better for the time being. I know there's more charts queued that will end up in this range so stuff will probably move a bit.

I'd rather move M8BT up than AT down. I don't think a gap of two difficulty points exists between those two charts which is why AT didn't also move down.

Punkture is Punkture. There's players that think it shouldn't even be a 95 but an overwhelming majority thinks otherwise. Scoring data is vomit and nothing else exists to compare it to. 97 at the min works, 98 is likely okay.

_.Pulse isn't here because I'm projecting a shift to 97. If people think 98 is more appropriate then sure.

I wanted to bring some 101s up to lessen how many charts exist in that tier but none of them feel as difficult as what's included in the tier above. I feel there's a hard line between the 101s and 102s. Also considered breaking up the 101s into two tiers and pushing 102+ to 103+ but the 101s truly feel quite similar in difficulty to me, and I don't think there's a two point gap between anything that would be considered a lower end 101 and the current 102s. Think we might just need more charts in the 102/103 range (DZ resubmit Apocynthion..)

Feedback appreciated.
__________________

Quote:
 Originally Posted by ilikexd i want to be cucked by cirno

Last edited by One Winged Angel; 08-28-2019 at 05:57 PM..

 Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home Flash Flash Revolution     FFR General Talk         Songs of the Week         Bug Reports and Suggestions         Contested Chart Difficulties     FFR Events         Tournaments         FFR Trading Card Game     FFR Batch Forum         Active Sets         Completed Sets         FFR Songs and Artist Permissions     Brag Board         Brag Board Archives     FFR Profiles General Discussion     Chit Chat         Birthdays         Ask Me Anything     Introductions     Critical Thinking         Homework & Help     Technology Gaming     Stepmania         Simulator Files     Other Music Games     The Werewolf Game         TWG Archives     Other Forum Games     Video Games Life and Arts     Art and Graphics     Audio and Music Production     Films and Television     Writing and Literature     Health and Fitness     Sports

All times are GMT -5. The time now is 06:51 PM.