Re: Skill Rating & Leaderboards Discussion [Updated Feb 17]
I think this curve falls off too steeply to accurately gauge a player's skill.
A) Why have it decrease all the way to 0? Why not cut it off at 75% at the 28g mark? Someone could get 28g on RATO and it be beaten out by a Free Space AAA.
Then again, someone could get 20g on RATO and have it beaten out by an 80 AAA
B) The curve simply drops off too steeply, adding 1 point of difficulty adds more than 1g to your score. This will underrate players who don't whore for AAA's and overrate players than do.
Ex. 16-18g on an 80 is about equal with a 64 AAA - just no way in hell those are in the same ballpark.
For instance in my own, I would consider the more difficult song here the better score -
#7 63 None Would Escape 6,243 1372 0 0 0 2 59
#8 67 Oops 6,237 1394 5 0 0 4 13
Adding 4 points of diffuculty should theoretically add more than 5-0-0-2 to your score on any given song.
C) This system overrates a player's strengths and completely neglects weaknesses. Then again, everyone has strengths and weaknesses. However this system really shows how good someone is at their strengths, and not a measure of all around skill. For example, my jumpstreaming ability is just plain bad compared to my scores on non-JS charts in the FMO range.
D) Suggestions - Fix the difficulties on many errant charts, especially in the lower range. The D2-D3 range is very off because of just a few charts (please fix party4u now that you can record a score without a background). Also because of this new skill system, I think there should be a mixed consideration for difficulty to AAA and Difficulty to SDG. I know OWA currently has spoken against this, but if our skill ratings are going to show this balance, why wouldn't the song difficulties to some extent? ABCDeath is a perfect example, and if it's showing up on many, multiple players top 4 songs, it's clearly overrated.
I like the system, but it can certainly use some tweaking. It doesn't need a major slope change as the difficulty/PA drops, but it needs to be propped up some at least.
What is Skill Rating?
Skill Rating is a new skill-based ranking system that assigns a rating to each player based on their performance on public songs. The following graph describes the relationship between the number of goods in a score and how well the difficulty of the song represents your skill:
This formula was created using data accumulated by the Official Tournament division placement team. Using this information, each qualifying score with less than 28 goods can be changed into a AAA-equivalent score, giving you the difficulty of the song on which you have essentially played with 0 goods. This difficulty is multiplied by one hundred to yield your rating.
Skill Rating is a new skill-based ranking system that assigns a rating to each player based on their performance on public songs. The following graph describes the relationship between the number of goods in a score and how well the difficulty of the song represents your skill:
This formula was created using data accumulated by the Official Tournament division placement team. Using this information, each qualifying score with less than 28 goods can be changed into a AAA-equivalent score, giving you the difficulty of the song on which you have essentially played with 0 goods. This difficulty is multiplied by one hundred to yield your rating.
A) Why have it decrease all the way to 0? Why not cut it off at 75% at the 28g mark? Someone could get 28g on RATO and it be beaten out by a Free Space AAA.
Then again, someone could get 20g on RATO and have it beaten out by an 80 AAA
B) The curve simply drops off too steeply, adding 1 point of difficulty adds more than 1g to your score. This will underrate players who don't whore for AAA's and overrate players than do.
Ex. 16-18g on an 80 is about equal with a 64 AAA - just no way in hell those are in the same ballpark.
For instance in my own, I would consider the more difficult song here the better score -
#7 63 None Would Escape 6,243 1372 0 0 0 2 59
#8 67 Oops 6,237 1394 5 0 0 4 13
Adding 4 points of diffuculty should theoretically add more than 5-0-0-2 to your score on any given song.
C) This system overrates a player's strengths and completely neglects weaknesses. Then again, everyone has strengths and weaknesses. However this system really shows how good someone is at their strengths, and not a measure of all around skill. For example, my jumpstreaming ability is just plain bad compared to my scores on non-JS charts in the FMO range.
D) Suggestions - Fix the difficulties on many errant charts, especially in the lower range. The D2-D3 range is very off because of just a few charts (please fix party4u now that you can record a score without a background). Also because of this new skill system, I think there should be a mixed consideration for difficulty to AAA and Difficulty to SDG. I know OWA currently has spoken against this, but if our skill ratings are going to show this balance, why wouldn't the song difficulties to some extent? ABCDeath is a perfect example, and if it's showing up on many, multiple players top 4 songs, it's clearly overrated.
I like the system, but it can certainly use some tweaking. It doesn't need a major slope change as the difficulty/PA drops, but it needs to be propped up some at least.
Comment