04-23-2016, 05:23 AM | #1 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Suggestion/Proposal: Tier Point Requirements Revamp
So, as I see it now, the tier point system is flawed, and here's why: It was intended so that getting X tier points on a song would be approximately the same difficulty for every song on a list, regardless of the difficulty of the song. As it stands now, this is simply not the case. For example, my score on Starlight[Macgravel] is ranked with an AAA-eq of 47.82, yet it gives only one tier point. If I were to get this same 47.82 on Seven (equivalent to a score of around 3-0-0-2), that would be worth 3 tier points.
So, here's my proposal to make this more in line with the original intention: Make every X tier point requirement be the same AAA-eq, rounded up to the next good. Here are my proposed AAA-eq cutoffs for each tier point: 1: Any Pass 2: 30.00 3: 40.00 4: 50.00 5: 58.00 6: 66.00 7: 73.00 8: 79.00 9: 85.00 10: 89.00 11: 92.00 12: 94.00 13: 96.00 14: 98.00 4-11 are AAAs of the lowest song with that tier point value, while 2-3 and 12-14 are extrapolations. For some examples of what this would do to requirements, here are what 5 songs would look like under the new system: Infernoplex (AAA = 72,750) 4/4: 72,725 (BF) -> 72,625 (5g) 3/4: 72,625 (5g) -> 72,150 (24g) 2/4: 72,450 (12g) -> 71,550 (48g) 1/4: Pass Exciting Hyper Highspeed Star (AAA = 66,850) 6/6: 66,850 (AAA) -> 66,700 (6g) 5/6: 66,750 (4g) -> 66,325 (21g) 4/6: 66,625 (9g) -> 65,675 (39g) 3/6: 66,350 (20g) -> 65,200 (66g) 2/6: 65,850 (40g) -> 64,350 (100g) 1/6: Pass Ketsarku Mozgalom (AAA = 81,750) 8/8 - 81,750 (AAA) -> 81,750 (AAA) 7/8 - 81,650 (4g) -> 81,425 (13g) 6/8 - 81,525 (9g) -> 81,050 (28g) 5/8 - 81,250 (20g) -> 80,550 (48g) 4/8 - 80,750 (40g) -> 79,975 (71g) 3/8 - 79,750 (80g) -> 79,100 (106g) 2/8 - 78,250 (140g) -> 78,000 (150g) 1/8 - Pass Almost There (AAA = 96,100) 10/10 - 96,075 (BF) -> 95,975 (5g) 9/10 - 95,975 (5g) -> 95,700 (16g) 8/10 - 95,800 (12g) -> 95,300 (32g) 7/10 - 95,475 (25g) -> 94,875 (49g) 6/10 - 94,850 (50g) -> 94,300 (72g) 5/10 - 93,600 (100g) -> 93,600 (100g) 4/10 - 91,725 (175g) -> 92,775 (133g) 3/10 - 89,850 (250g) -> 91,550 (182g) 2/10 - 87,350 (350g) -> 89,975 (245g) 1/10 - Pass Death Piano (AAA = 122,650) 14/14 - 122,625 (BF) -> 122,575 (3g) 13/14 - 122,525 (5g) -> 122,425 (9g) 12/14 - 122,350 (12g) -> 122,250 (16g) 11/14 - 122,025 (25g) -> 122,100 (22g) 10/14 - 121,400 (50g) -> 121,850 (32g) 9/14 - 120,150 (100g) -> 121,525 (45g) 8/14 - 118,275 (175g) -> 121,000 (66g) 7/14 - 116,400 (250g) -> 120,425 (89g) 6/14 - 113,900 (350g) -> 119,700 (118g) 5/14 - 110,150 (500g) -> 118,750 (156g) 4/14 - 106,400 (650g) -> 117,675 (199g) 3/14 - 102,650 (800g) -> 116,075 (263g) 2/14 - 97,650 (1000g) -> 114,000 (346g) 1/14 - Pass As you can see, for songs worth up to 8 tier points, every requirement is relaxed, some by a significant amount. For harder songs, lower requirements are tightened and higher ones are relaxed, and for the very hardest songs, most of the requirements are tightened. So that's my proposed system in a nutshell. Now, I'll cover a few points/concerns some of you may have upon seeing this new system. -"This would devalue AAAs!" Not at all, the extra point bonuses for getting a lot of AAAs would remain the same. -"This would make lower tiers too easy to get!" On the contrary, I think this is a favorable outcome, as it opens the tier point system to players of lower skill. At the moment, you must be at least low-mid D3 to really begin to use the tier point system (aside from the 1/X pass requirements), whereas under the new system, players as low as mid-D2 would be able to gain some 2/X points. -"The requirements for the hardest songs are too hard!" This, I think, is because most of the hardest songs in the game are not adequately represented at their current ratings, and should be moved up for proper standing. [And as a side note, doing this has completely changed my tune on expanding to a 1-120 scale: now I'm not just for it, I think it's absolutely essential.] Death Piano, for example, is absolutely not suitable at 99--it should, properly, be rated about 115. Getting an AAA-eq of 30.00 on Infernoplex is much, MUCH easier than getting an AAA-eq of 30.00 on Death Piano. If all songs were rated perfectly, and getting an AAA-eq of X was exactly the same difficulty for every song, this problem wouldn't exist, but I understand that doing this is nigh impossible. So, that's my proposal. If you have questions or concerns, feel free to ask. It's super super late at night and I'm going to bed, so I'll answer stuff tomorrow morning.
__________________
Last edited by RenegadeLucien; 04-23-2016 at 05:25 AM.. |
04-23-2016, 01:57 PM | #2 |
Under the scarlet moon
Join Date: Jan 2014
Age: 31
Posts: 921
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Only problem I see is that the AAA-eq formula sucks for anything over a few greats.
|
04-23-2016, 02:15 PM | #3 |
Anime Avatars ( ◜◡^)っ✂╰⋃╯
Join Date: Mar 2007
Location: Squat Rack
Age: 34
Posts: 10,837
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Even for scores in good ranges closer to AAA, the skill rating algorithm is inaccurate.
I appreciate the amount of thought you put into this, and I'm not claiming the tier point setup is currently perfect (though a lot of thought went into establishing which songs would award X points). Numerous issues arise though, the most prominent one being that the skill rating algorithm doesn't accurately reflect AAA equivalency. Take two songs, one rated 94 and the other a point above at 95. Most D7 players would probably agree that a BF on the 95 is equally as impressive as a AAA on the 94 (and perhaps maybe even less impressive, as a 1g difference when trying to improve to AAA is astronomically more difficult to shave off than going from 10g to 9g, or even 3g to 2g). The difference in an additional difficulty point is vastly overestimated in the algorithm. A score of 2-0-0-2 on the 95, for example, receives an AAA equivalency value of 94.XX, meaning the algorithm reads this score as more impressive than any 94 AAA. I know the playerbase wouldn't agree with that comparison. Furthermore, the skill rating formula decreases AAA value linearly (I believe) for scores in these good ranges, which shouldn't be the case, and the stat for that 95 score becomes increasingly overvalued as more goods accumulate (at least in comparison to files in that general difficulty range, comparing a 95 to something in the 50 difficulty range is an entirely different beast). I also would not want to set the tier max for any one song to a score value lower than a BF. Maxing a song's tier points implies mastery or near mastery of that song, and setting 2g or lower as the upper bound doesn't imply that. Other problems exist on the lower end as well as Autotelic pointed out but I'm far too lazy to get into those intricacies and why imo those scores do not display an accurate reflection of AAA equivalency. I believe changes are in the works for another more accurate iteration of the skill rating formula but I'll let an admin/dev comment on that. Last edited by One Winged Angel; 04-23-2016 at 02:29 PM.. |
04-23-2016, 04:59 PM | #4 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Yes, I can see that my proposed system is extremely dependent on the skill rating formula, and if that formula is flawed (as you two claim it is), the system will be flawed too. I personally put more trust in the formula than you two do, but then again I'm far less skilled than you two so what do I know.
However, if the formula is as flawed as you say it is, that brings up a whole lot of other questions, since FFR rank and divisional placement are just as dependent on the formula as this system is, and perhaps even more so, so the accuracy of those two metrics are immediately thrown into doubt. I currently have a skill rating of 44.5, and D3 cutoff is 46, so am I D2 because I actually lack the skill to be D3 or because an inaccuracy in the formula misrepresents my skill, and I actually do have the skill to be D3? With all this in mind, an update to the formula, one that more of the community can agree with, should probably be done. And if the new formula accurately maps scores to their AAA equivalents, that would fix most or all of the problems pointed out with the system. As for setting the max tier points for one song to be a BF/AAA, I'm going to have to disagree on that one. I'm sure you'd agree with me that 2g-ing a 57 is considerably harder than AAA'ing a 50, so giving the AAA 4 points and the 2g 3 points just seems incorrect to me.
__________________
|
04-23-2016, 07:05 PM | #5 | ||
Under the scarlet moon
Join Date: Jan 2014
Age: 31
Posts: 921
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Quote:
From what I remember, the current formula idea grew at first from some tool that analyzed levelranks to grab highest AAA, highest SDG and things like that. Like that tool, the formula output is simply an additional helper in analyzing players. The less bad ranking system was just a nice bonus. Quote:
|
||
04-23-2016, 07:15 PM | #6 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Hmmm...really? I guess my own experiences are atypical, then. Personally, my high AAAs are 35, 34, and 32, and I only have one sub-5g score above a 40, so for me, at least, once you CAN get a low SDG, getting an AAA isn't that big a step up.
In fact, I've always been of the opinion that if you're capable of a low SDG, you're capable of an AAA (barring mental blocks or some other limiting factor.)
__________________
|
04-23-2016, 07:43 PM | #7 | ||
Retired Staff
All the things
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Quote:
Quote:
|
||
04-24-2016, 02:15 AM | #8 | |||
Mostly Ignored
Skill Rating Designer
Join Date: Mar 2011
Posts: 471
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Quote:
It's been challenging to make a single formula that adheres to the idiosyncrasies of our difficulties. It is informed by the best wisdom of the individuals we surveyed, but to some extent cannot adapt very well to the "non-smooth" reality of difficulty. Quote:
Oh, and there will have to be a new version when/if the new scale happens, so we're pretty much waiting for that. Quote:
If you'd like to make a stronger case, it would be helpful to point out which discrepancies are the most egregious, and then I suppose we can decide which system is correct.
__________________
2014 October 7th 1:03 AM Zageron: Trumpet Trumpet63: yes, im here Zageron: You have a problem. |
|||
04-24-2016, 02:26 AM | #9 |
FFR Player
Join Date: Oct 2015
Location: Banda Aceh, Indonesia
Age: 34
Posts: 57
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
What do "Tier Points" even mean? AAA and FC bars indicate the number of files you managed to AAA/FC but what is a "Tier Point" bar supposed to indicate?
|
04-24-2016, 02:42 AM | #10 | |||
🡸Index🡻Is🡹Fun!🡺
Join Date: Dec 2010
Location: Auserd
Posts: 1,924
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
how many tier points you have
__________________
Old Quotes Quote:
Quote:
Quote:
|
|||
04-24-2016, 02:50 AM | #11 |
Mostly Ignored
Skill Rating Designer
Join Date: Mar 2011
Posts: 471
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
There isn't really a deeper meaning... they're arbitrary points awarded for doing well on certain songs.
__________________
2014 October 7th 1:03 AM Zageron: Trumpet Trumpet63: yes, im here Zageron: You have a problem. |
04-24-2016, 03:40 AM | #12 |
Sir Krisk
Join Date: Dec 2007
Location: Vic, Australia
Posts: 930
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
I do like the idea of opening the tier point system to lower level players as the site needs something to keep new players interested, other than just FCs and AAAs. More lower level songs could be added as well.
For people wanting to keep the tier point system for higher level players only, maybe there could be a tier point system based on rates. I feel this could be a huge expansion. Also, on the topic of rates, why does it now say under 'Recently Played', the 'rate' on which the song was played? What is the point of this if rate scores aren't saved anyway?
__________________
|
04-24-2016, 04:12 AM | #13 | |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Quote:
Well, here's an example of the AAA-eq ratings of the current tier point requirements of lower and upper bounds of each tier point level: The two main problems with the current formula are immediately obvious: 1. Near-AAA (but sub-AAA) scores are rated too high, and 2. High-good scores are rated too low.
__________________
Last edited by RenegadeLucien; 04-24-2016 at 05:54 AM.. |
|
04-24-2016, 08:51 AM | #14 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
So...with a lot of help from various tools, I have managed to draft a new version of the AAAeq skill rating formula that is based on the tier point list, and (hopefully) more accurate towards near-AAA and high-good scores.
The formula is as follows: AAAeq = (D-126.406)*((1 + (NGC/3.429871)^1.151702)^0.1682445) + 126.406 where D = the difficultly of the song, and NGC is the number of goods in the score to be ranked (where Average = 1.8 goods, Miss = 2.4 goods, and Boo = 0.2 goods.) (NOTE: This formula alone will rank a lot of scores with a negative AAAeq value. I worked around that by assigning scores to the maximum of the above formula and the formula (1-NQC/1000). This will only matter if the score in question is ranked by the above formula to be below 1.00.) Obviously that's just a mess of numbers and symbols, and doesn't tell you anything. Everyone loves graphs, so let's add some graphs: And here's the original, for comparison: As you can see, the new formula drops off far faster than the old one, but also maintains a higher score for longer on harder songs. (Of course, I'm not suggesting this formula is perfect by any means...despite my attempt to patch this, there exists a huge hole for low D1 players. Maybe someone else can come up with a way to plug this hole.)
__________________
Last edited by RenegadeLucien; 04-24-2016 at 08:54 AM.. |
04-24-2016, 11:57 AM | #15 |
FFR Player
Join Date: Dec 2007
Location: nima
Posts: 4,278
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
hey im going to do that thing where it doesnt look like im complimenting you but really i am
you're not dumb, but you are wrong |
04-24-2016, 02:38 PM | #16 | |
[Nobody liked that.]
Join Date: Sep 2012
Posts: 3,355
|
Suggestion/Proposal: Tier Point Requirements should be deleted
Quote:
Let's get rid of it for adding nothing of real value. There is zero things tier points do that isn't more useful than what we already have. I for years have never understood why this system has been kept, or implemented. Last edited by XelNya; 04-24-2016 at 02:38 PM.. |
|
04-24-2016, 09:30 PM | #17 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
I thought people liked the TP system? There was at least enough interest in it to revamp it once a few years ago...maybe times have changed.
But in any case...more time, more tools, new skill formula, draft 2! This time without huge holes...at least that I can find. AAAeq = DIFF * (a + (1 - a)/(1 + (NGC/c)^b)), where: a = -27861.2 + (0.0023176+27861.2)/(1 + (DIFF/977.0595)^5.396388) b = 0.0005334*DIFF^2 - 0.086575*DIFF + 4.228044 c = 384644700 + (30.29862-384644700)/(1+(DIFF/213.2135)^16.80688) NGC = good count, and DIFF = song difficulty. New graphs: I feel a LOT better about this formula...the negative score issue is resolved, and it is much better with high-good scores than last time.
__________________
Last edited by RenegadeLucien; 04-24-2016 at 10:43 PM.. |
04-24-2016, 10:20 PM | #18 | |
Fractals!
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Quote:
Keep up the good work, though! |
|
04-24-2016, 10:30 PM | #19 |
Mostly Ignored
Skill Rating Designer
Join Date: Mar 2011
Posts: 471
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Do you mean 1/(1-a) instead of (1-a)/1 ?
__________________
2014 October 7th 1:03 AM Zageron: Trumpet Trumpet63: yes, im here Zageron: You have a problem. |
04-24-2016, 10:42 PM | #20 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Nah, I left out a parentheses. I meant DIFF * (a + (1 - a)/(1 + (NGC/c)^b))
__________________
|
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Thread Tools | |
Display Modes | |
|
|