Flash Flash Revolution

Flash Flash Revolution (http://www.flashflashrevolution.com/vbz/index.php)
-   Contested Chart Difficulties (http://www.flashflashrevolution.com/vbz/forumdisplay.php?f=76)
-   -   Standardizing Difficulty Ratings (http://www.flashflashrevolution.com/vbz/showthread.php?t=152655)

WirryWoo 12-15-2020 09:04 PM

Standardizing Difficulty Ratings
 
Hey all. I know there has been some previous conversations about standardizing the definitions of what it means for a file to be rated difficulty x. I thought it might be a good idea to consolidate all of our thoughts into this thread to keep a transparent understanding of why a new file is rated y, and fix some of the current difficulty ratings in game.

I'm not exactly sure how to devise a good system to update and track all of that information but my thinking would be something along the lines of anyone posting thoughts about various files. I'll update (try my best to at least) this post with pending ideas, and we'll let a difficulty consultant accept or reject the ideas. We'll store this finalized list here to keep a record of what was previously discussed.

Note: More than one file can officially take the same difficulty rank. Try to keep this shared difficulty idea to a minimum though, so we can make accurate comparisons for future files. This list will continually iterate over time.

Official Rankings:
[insert difficulty and song]

Awaiting for Approval:
90 - Double Helix
100 - Rave 7

I started this list by adding Double Helix as a solid 90. I think the variety of patterns present in the chart are shared across other files currently rated 90. We had previous conversations about this, but I'm happy to continue that conversation here alongside with other suggestions others might have.

Thanks!

FlynnMac 12-15-2020 09:08 PM

Re: Standardizing Difficulty Ratings
 
Interesting idea! My only question is to why we only establish one file at any given difficulty, as files within the same difficulty tend to cater to different skills, i.e. Jacks, Tech, and Jumpstream. I just need a little bit of clarity on that part.

WirryWoo 12-15-2020 09:15 PM

Re: Standardizing Difficulty Ratings
 
Quote:

Originally Posted by FlynnMac (Post 4752733)
Interesting idea! My only question is to why we only establish one file at any given difficulty, as files within the same difficulty tend to cater to different skills, i.e. Jacks, Tech, and Jumpstream. I just need a little bit of clarity on that part.

Yep good question. The reason being is because the goal of having these standardized difficulties is to make fair comparisons for future files. Having too many of them defeats this purpose. I understand that more than 1 file can do a better job at "defining" the difficulty requirements for each rating, so I leave that as an option if a difficulty rating comes down to that case.

Andrew WCY 12-15-2020 09:54 PM

Re: Standardizing Difficulty Ratings
 
I think this is a good idea. Determining what charts are good representatives of various difficulties helps giving files difficulty ratings easier as we'd have standardized points of reference.

With that being said, I wholly agree with you on Double Helix being a solid 90. Let's get that list filled in.

EDIT: I wonder what others (including Difficulty Consultants) think about this!

Bolth mannn 12-16-2020 12:22 AM

Re: Standardizing Difficulty Ratings
 
perhaps we could have one file represent each of the specific patterns that you find at each level?

someone good at jumpstreams will find certain level 70s easier than others because of jacks or whatever

Blackskull305 12-16-2020 12:27 AM

Re: Standardizing Difficulty Ratings
 
I always hear people say that Rave 7 is the best model of a solid level 100 file.

Dynam0 12-16-2020 08:15 AM

Re: Standardizing Difficulty Ratings
 
Oh cool, I have a ton of work to do here today but try to shoot some thoughts when I get home.

WirryWoo 12-16-2020 04:23 PM

Re: Standardizing Difficulty Ratings
 
Quote:

Originally Posted by Blackskull305 (Post 4752747)
I always hear people say that Rave 7 is the best model of a solid level 100 file.

Updated the OP. I too heard something similar, so I think it's fair to add that to the list. Thanks!

Dynam0 12-16-2020 06:25 PM

Re: Standardizing Difficulty Ratings
 
As difficulty is subjective you do need a starting point and we have a pretty good one with the current difficulty consultants but there aren't nearly enough of them.

The proper way of doing this would be to assign a range of difficulties to groups of players with a given skill rating "X" to judge. The range of difficulties a player should judge would be X ± 10 although this number could vary depending on how anal you want to be.

Take me for example. My skill rating is 99.58 (round it to 100) thus I ought to be able to judge charts accurately within the 90-110 range from easiest to hardest. Of course I have some bias towards some charts over others, that's why more players assigned to each interval are necessary.

From there you would order charts within the ranges from easiest to hardest. The question then becomes where does a 100 turn into a 101 etc. which is why the standards in the OP are necessary.

The above is the ideal case and I know we haven't the number of volunteers to manage this task so we're left with a two-step solution which I believe is the current one:

1) A smaller group of players judge charts, some of the charts will be outside of their interval of competence. Also, there would be more bias as there are less judges.

2) After a chart is in-game for a time and enough scoring data is accumulated, difficulties can be fine-tuned to fit to the data.


All the above being said, I will give this some thought and perhaps make a list of standards but I do think there are some qualities that ought to be considered when choosing:

-Song length (avoid anything too long or too short)
-Difficulty spikes (these are a huge no-no as then the chart is prone to flukey scores)
-Pattern bias (avoid charts that focus on one skill only, jacks, trills, rolls, etc.)
-Stamina (avoid charts that obtain most of their difficulty from being draining. This one is tricky but I think a good example is Hyper Weaboo Vodka Shots [100]. If you look at the scores for this one, you'll note that they drop off severely with skill rating and it's due to stamina. It's hard to articulate but I hope that makes sense.)

WirryWoo 12-16-2020 06:47 PM

Re: Standardizing Difficulty Ratings
 
Damn Dynam0, our minds think so similarly LOL.

Completely agree with your entire post. I think that will eventually raise the question about rating future files that don't meet the standards mentioned above. For example, Almost There, do we rate the file based on the 32nds, or the entire file? There are pros and cons to both choices but I think the overall preference in the community seems to be the former.

Only thing I dislike about the first option is the ease to farm free AAA equivalency from grinding that file. If you play the file enough times, you'll soon score a good PB and get free equivalency off of it, but I think that's mainly the fault of the skill rating formula more than the difficulty of the file itself.

Overall, good standards and awesome post.

T-Force 12-17-2020 07:31 PM

Re: Standardizing Difficulty Ratings
 
I hope this thread gets some traction because I think this could definitely be beneficial.

Andrew WCY 12-17-2020 08:40 PM

Re: Standardizing Difficulty Ratings
 
I have some suggestions to add to the list:

90: Milky Blue, Tenimuhou
91: Love Hurts (Lonely Dance Mix), Storm Raid Battle
92: The Limit Does Not Exist, Canary Part V
93: Phi-dentity Crisis, The Glorious (Final Sketch Remix), Forgotten
94: Rave 5, Extratone Pirates, Longinus
95: Japan Style Breakcore!!!, grind4
96: Counter Clockwise Chant Pattern, Slashmaid (instrumental), Grist
97: You're gonna love my nuts, All your base are belong to ME, Heavenly Castle velier
98: Playing Super Mario World While Taking Mushrooms, A Quick Death, La Campanella
99: Stress Free Style, Call me it. (500 Tortures), to luv me i *** for u
100: oppIrish (GYPSYCOREremix), where is my balls, Jamais Vu

What do you all think about this? Do you agree with these suggestions? What other suggestions do you have in mind?

EDIT: I am aware that we should try to minimize the number of files used as standards for each difficulty, but I think sometimes it's useful to keep more than 1 point of reference as not all charts are the same (jumpstream-oriented, jacks, technical files etc.). This could make it easier for us to judge whether a certain kind of stepfile belongs to a specific difficulty rating later on, since we have some charts of the same/similar type that are upheld as difficulty rating standards.

hi19hi19 12-18-2020 12:45 AM

Re: Standardizing Difficulty Ratings
 
For a while Reality was "the 85" when it came to stream speed, not sure if it's still viewed as a difficulty benchmark anymore?

Interesting thread to be sure.

Travis_Flesher 12-19-2020 08:09 AM

vertex BETA is not 68, more like 82
 
my scores can be used for calculating. 2415/2581 songs played. bu jiao
all played exactly the same. legit and as perfect as i can, always on thin-key laptop, bu jiao
carefully mashing to maintain combo when needed, never change offset or speed. quite standard.
i need to log in and top out some unplayed ones, which are probably the main ones of the topic, but yeah. I would trust me.
but yeah, this thread is prolly the best way to do it. discussing the files, comparing scores. but yeah..
i would assume that comparing score sets from players like me, intermediate+ level, consistent, and not all AAAs is the way.
dunno anybody right off the bat. but ya, i'd absolutely not really put very much consideration into AAAs and goodflags as factors
when determining a standard difficulty for a popular game enjoyed by numerous demographics.
but no, i am aware that in cases of high level play and competitiveness, it's different. there's lots of variables and nuances yknow.
i mean, i play ffr like Goku training to destroy Cell, and it still bests me every time. ight peepz, stay beautiful, hmu


All times are GMT -5. The time now is 08:31 AM.

Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright FlashFlashRevolution