08-26-2019, 10:16 PM | #21 |
✘ Forever OP✘
Join Date: Dec 2008
Location: Canada,Quebec
Age: 29
Posts: 4,171
|
Re: Scoring Data in Brutal Difficulty Range
Your idea of variable decay is basically something I brought up in chitchat discord but less extreme; the ideal formula would be one that varies for each chart.
Let f(c, s) = a Where c is a chart, s is the raw goods score and a is the resulting AAA equivalency. This gives much flexibility over various chart structures and can be implemented without too much trouble. Of course, that requires some computed difficulty, hence my work on that some time ago. I'll see if I can extract some useful stats for this thread this week. I dont think the current system is "fixable" without big changes, it simply does not account for chart structure, which it should. |
08-26-2019, 10:27 PM | #22 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 28
Posts: 282
|
Re: Scoring Data in Brutal Difficulty Range
Well, the current system relies entirely on the manually-assigned difficulty rating to sum up all the factors like chart structure. To do anything different, we'd basically need an automatic difficulty calculator like Etterna's.
__________________
|
08-26-2019, 10:37 PM | #23 | |
✘ Forever OP✘
Join Date: Dec 2008
Location: Canada,Quebec
Age: 29
Posts: 4,171
|
Re: Scoring Data in Brutal Difficulty Range
Quote:
Although Rob's idea would be a patch for the apparent edge cases, at some point if we want accurate difficulty we need better maths, that's just how it is. I got so much shit for trying to make a calc, yet it's clearly the way to go, especially for a game like this with so many charts. Edit: The manual difficulty can only be the AAA difficulty. The AAA equivalency formula computes a score for any raw goods count. It is evident that the mapping makes 0 sense. Last edited by xXOpkillerXx; 08-26-2019 at 10:42 PM.. |
|
08-26-2019, 10:39 PM | #24 | |
Anime Avatars ( ◜◡^)っ✂╰⋃╯
Join Date: Mar 2007
Location: Squat Rack
Age: 34
Posts: 10,837
|
Re: Scoring Data in Brutal Difficulty Range
Quote:
Scores on RATO/DP from D6/D7 players were spitting out equivalencies in the FMO or lower range for years but no one paid any mind because it was just a couple charts and sure whatever that's fine I guess. But now that there's gonna be more and an entire division is going to be reliant on most of that range for what comprises their skill rating, that definitely needs to change. Last edited by One Winged Angel; 08-26-2019 at 10:43 PM.. |
|
08-26-2019, 10:54 PM | #25 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 28
Posts: 282
|
Re: Scoring Data in Brutal Difficulty Range
Who gave you shit aside from Mina? I don't think anyone here would actively oppose you or anyone else trying to make a difficulty calc.
It's been stated over and over and over again by so many people that the current system's accuracy drops very fast as you go higher up in good count; I don't think this was ever an issue unique to DP/RATO or even high difficulty songs in general.
__________________
|
08-26-2019, 11:15 PM | #26 | |
Anime Avatars ( ◜◡^)っ✂╰⋃╯
Join Date: Mar 2007
Location: Squat Rack
Age: 34
Posts: 10,837
|
Re: Scoring Data in Brutal Difficulty Range
Quote:
I don't see an issue with trying to hammer out any and all issues in an effort to create a more accurate system. I feel like you take these comments as personal attacks and are quick to displace blame elsewhere, such as on the difficulties having needed to account for this when this was a system assumptive of numerous chart qualities being meticulously considered and represented by a single number so as to treat them identically when extrapolating a score's worth. |
|
08-26-2019, 11:55 PM | #27 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 28
Posts: 282
|
Re: Scoring Data in Brutal Difficulty Range
I'm not sure where I gave off the impression that I'm taking any of this personally, but if that's what you're getting, that wasn't my intention, I'm sorry for giving off that impression. The current system isn't even mine.
I'm pretty sure we're on the same side here. The system needs to change. My point is that the only way we're really going to get something that's truly accurate is with a calculator. Yeah, we could have some sort of variable decay rating for each song that tunes the base skill rating formula for it, but to do that we'd need a calculator anyway, unless someone wants to manually go through all 2000+ songs and give all of them another number.
__________________
|
08-27-2019, 07:10 AM | #28 | |
✘ Forever OP✘
Join Date: Dec 2008
Location: Canada,Quebec
Age: 29
Posts: 4,171
|
Re: Scoring Data in Brutal Difficulty Range
Quote:
Anyway, if interest for a calc goes up for real now, I might have some motivation to help. Last edited by xXOpkillerXx; 08-27-2019 at 07:24 AM.. |
|
08-27-2019, 07:54 AM | #29 |
✘ Forever OP✘
Join Date: Dec 2008
Location: Canada,Quebec
Age: 29
Posts: 4,171
|
Re: Scoring Data in Brutal Difficulty Range
Here's my take on the difficulty factors of individual notes:
-Local one-hand complexity: how difficult it is to hit that note given the past and future X notes on the same hand (future notes are necessary to account for readability). That is something I havent finalized, but generally the difficulty goes up first with the spacing of notes (the less frames between the notes, the harder it is in a non-linear fashion so that 1-framers are much harder than 2-framers but 30 and 31 frames are pretty similar), then by transition (at the same speed, a jump to a single note is always harder than a minijack or a jumpjack, and single-to-single like 12 or 34 or 21 or 43 have a special weight for being easily hit as a jump or not). Some time-based gaussian window over each note gave decent results with a window of about 1 second (30 frames) or less on each side and a low std dev (< 1.0). -Global 2-hands complexity: a distribution of the two 1-hand complexities at each timestep. For example, a very hard section on one hand with a very simple one on the other hand could be easier than medium difficulty on both hands at same time. This would need refinement for polys and a more well-defined explanation (with general and edge cases). -Note time: just a factor of where the note is in time. This has to be picked/formulated so that a note after 5 minutes with low complexity cannot be harder than a note 1 minute in with high complexity. It is easily defined once the complexities are defined. Accounts for focus loss and partly for stamina. -Note stamina: a large time-based past-only window over the aggregated factors above. Essentially accounts for breaks in a song; it's easier to hit a hard section after a break than in the middle of some stream or whatever. This gives a difficulty number to each note of a file. It then becomes possible to compute a different AAA equiv formula for each file. The overall difficulty of a file would then not be a single number, but rather a distribution over raw goods count. Nothing forbids us to compute and show difficulty for a specific count (AAA difficulty, 10g difficulty, 20g difficulty, etc). I have a pretty good setup to compute these already, so I'm saying it here to gather more opinions on the aggregation part and various factors (gaussians parameters, complexity, etc). @rob if you prefer this to be in another thread let me know Last edited by xXOpkillerXx; 08-27-2019 at 08:10 AM.. |
08-27-2019, 10:37 AM | #30 |
The Dominator
Join Date: Sep 2005
Location: North Bay, ON
Age: 34
Posts: 8,987
|
Re: Scoring Data in Brutal Difficulty Range
My main concern with that approach is pattern manipulation and stamina. Getting those right would be a tough ask imo. I still think it is far less tedious to have subjective difficulty assignments and as Rob said we just need to get the decay part correct. It's not incredibly far off at this point.
|
08-27-2019, 10:42 AM | #31 | |
✘ Forever OP✘
Join Date: Dec 2008
Location: Canada,Quebec
Age: 29
Posts: 4,171
|
Re: Scoring Data in Brutal Difficulty Range
Quote:
PS: I already have computed the manipulatable sections mostly. That's dealt with by accounting for the number of frames you have to hit singles as jumps (1 frame being harder to manip, 2 frames is most likely a jump). As for stamina, I explained a basic framework for it; what would you disagree with ? Edit: I also disagree with "it's not incredibly far off"; far off what ? The only thing the current system can hope to reach is optimal subjective consensus (as in the most people who agree with the difficulties). While that isn't a bad metric per se, it is inevitably flawed and biased, and doesnt solve the problem: variable decay is still just a decay, it doesnt fully compensate for chart structure, it just covers cases where the distribution of ordered note difficulty ressembles one unique function with a modifyable decay. The truth is there can be many more shapes to that distribution. Last edited by xXOpkillerXx; 08-27-2019 at 11:03 AM.. |
|
08-27-2019, 11:28 AM | #32 |
The Dominator
Join Date: Sep 2005
Location: North Bay, ON
Age: 34
Posts: 8,987
|
Re: Scoring Data in Brutal Difficulty Range
Idk man I just think taking this granulated an approach to difficulty is not worth the effort and it's always going to be prone to outliers reliant on fudge-factoring. Who determines how difficult a trill is a 240bpm? Someone who is good at them? The average player? Who is an average player? You see how silly the concept of automating this is? It's entirely subjective. There will still be 97s that feel like 96s and so on based on individual player strengths.
|
08-27-2019, 11:40 AM | #33 | |
✘ Forever OP✘
Join Date: Dec 2008
Location: Canada,Quebec
Age: 29
Posts: 4,171
|
Re: Scoring Data in Brutal Difficulty Range
Quote:
|
|
08-27-2019, 11:45 AM | #34 |
The Dominator
Join Date: Sep 2005
Location: North Bay, ON
Age: 34
Posts: 8,987
|
Re: Scoring Data in Brutal Difficulty Range
Well considering my prior post stated that going into said details is not worth the effort, I am out of this discussion :P
Best of luck though, if you do figure out a way to do it then I'll be proven wrong and then some |
08-27-2019, 02:29 PM | #35 |
The Frog
Join Date: Sep 2007
Posts: 165
|
Re: Scoring Data in Brutal Difficulty Range
Correct me if I'm wrong, but isn't it quite rare to have an AAA equal to or above one's rating? The algorithm is excellent as far as I'm concerned -- so long as the song difficulties are correct.
Last edited by SputnikOwns; 08-27-2019 at 02:30 PM.. |
08-27-2019, 02:44 PM | #36 | |
🍍Pineapple Man🍍
Join Date: Nov 2017
Location: Pacific Timezone, USA Age: 22
Posts: 506
|
Re: Scoring Data in Brutal Difficulty Range
Quote:
shit like this happens but only because of how absurdly low the difficulty of this file outside of two or three main hard parts is compared to what it is at currently Edit: It is very likely that White Walls Part 2 is also seeking a nerf not much in AAA difficulty but rather the How-Easy-This-Can-Be-Abused factor which is the overall discussion we are having at the moment Last edited by Matthia; 08-27-2019 at 02:48 PM.. |
|
08-27-2019, 03:07 PM | #37 |
✘ Forever OP✘
Join Date: Dec 2008
Location: Canada,Quebec
Age: 29
Posts: 4,171
|
Re: Scoring Data in Brutal Difficulty Range
|
08-27-2019, 03:40 PM | #38 |
The Frog
Join Date: Sep 2007
Posts: 165
|
Re: Scoring Data in Brutal Difficulty Range
That's totally fine of course. They should be ordered correctly according to top players. No need to adjust the algorithm for 100+ though.
|
08-27-2019, 03:45 PM | #39 |
✘ Forever OP✘
Join Date: Dec 2008
Location: Canada,Quebec
Age: 29
Posts: 4,171
|
Re: Scoring Data in Brutal Difficulty Range
This is the second problem. As it was said a few times, there are a bunch of files which are hard AAAs but easy SDGs, and that breaks the current system.
|
08-28-2019, 04:40 PM | #40 |
Anime Avatars ( ◜◡^)っ✂╰⋃╯
Join Date: Mar 2007
Location: Squat Rack
Age: 34
Posts: 10,837
|
Re: Scoring Data in Brutal Difficulty Range
I agree with Dynamo, honestly there are far too many variables to deal with to ensure a calculator will work effectively. Manually establishing separate decay formulas for charts that are more accessible to less skilled players will reach higher levels of accuracy and communal consensus without the hassle of dealing with all the intricacies needed to be taken into account for a calculator in a game that encodes charts at 30fps. A 16th roll at 300bpm starting on a quarter note can effectively be jumptrilled because the one frame gaps all coincide between arrows hit with the same hand (1/2 and 3/4), but offset the roll by a 16th and now all the one frame gaps appear at 2/3 and 4/1. Having to account for quite literally the same pattern being variable in difficulty depending on the starting note, on top of everything else that needs to be taken into consideration...I personally wouldn't want to invest any time in that.
What Dynamo was fudging around with in Excel looks interesting and I might mess around with that later. Tangential to that, I've looked at suggestions and scoreboard data and am interested in feedback for this structure of 98+: A few notes: * denotes charts that would benefit from separate equivalency decays. I'm not suggesting they should share identical formulas, but a manipulation of the current formula is required. La Camp's scoring data after a decade in game suggests it can hang with the 99s. Structurally I may not entirely agree with that shift but it fills out the 99s a bit better for the time being. I know there's more charts queued that will end up in this range so stuff will probably move a bit. I'd rather move M8BT up than AT down. I don't think a gap of two difficulty points exists between those two charts which is why AT didn't also move down. Punkture is Punkture. There's players that think it shouldn't even be a 95 but an overwhelming majority thinks otherwise. Scoring data is vomit and nothing else exists to compare it to. 97 at the min works, 98 is likely okay. _.Pulse isn't here because I'm projecting a shift to 97. If people think 98 is more appropriate then sure. I wanted to bring some 101s up to lessen how many charts exist in that tier but none of them feel as difficult as what's included in the tier above. I feel there's a hard line between the 101s and 102s. Also considered breaking up the 101s into two tiers and pushing 102+ to 103+ but the 101s truly feel quite similar in difficulty to me, and I don't think there's a two point gap between anything that would be considered a lower end 101 and the current 102s. Think we might just need more charts in the 102/103 range (DZ resubmit Apocynthion..) Feedback appreciated. Last edited by One Winged Angel; 08-28-2019 at 04:57 PM.. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|