04-25-2016, 02:16 AM | #21 |
Administrator
User Administrator
Join Date: Dec 2007
Location: Vancouver, BC
Age: 29
Posts: 3,902
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Thank you RenegadeLucien for putting so much thought and effort into creating this proposal! I also really appreciate you justifying all of your suggested changes.
I think it is important to discuss what tier points mean to the community, so we can decide what direction we want to evolve this metric. The curated list of songs and their requirements were hand-picked with love. Understanding this, would applying a generalized formula to all songs of a particular difficulty break the "spirit" of tier points? What about using a generalized formula for every song in-game to create an "improved" average rank? The tier points system is unique to any other ranking metric. If we want to improve the accuracy of tier points, where do we stop before it becomes a tooltip to the skill rating algorithm? Tier points were introduced as a "ranking system for the top active players" at a time when players were ranked by grandtotal. If tier points are an outdated skill ranking system, and skill rating exists, is there a reason to keep both metrics? I feel the suggestions made in this thread could be better directed to improve the skill rating algorithm itself. |
04-25-2016, 02:31 AM | #22 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
First off, you're welcome, Prawn
As XelNya stated, tier points might have become outdated as a ranking system when the skill rating algorithm was released. However, I don't think they've become completely obsolete, because they were indeed "hand-picked with love" as you stated. Since the intention of the tier point system was to have X tier points on a song be similar in difficulty on every song, and the skill rating algorithm in its current form does not reflect this, perhaps the tier points can be used as a kind of calibration tool for the skill rating algorithm, as I have attempted to do. That being said, my new formula is far from flawless. Calibrating by tier points has an inherent flaw--there is no hard calibration data below difficulty 50, and only superficial data above difficulty 90. Therefore, extensive extrapolation is required to make the formula reach all songs, which inevitably results in errors. My formula's accuracy starts to degrade below 45, and completely falls apart below 30, in the near-AAA scores. (For instance, a 10g on a 15 would be ranked under my formula as a 14.5, which is ridiculous.) I could try tinkering with the constants, but this clearly needs additional work before it can become a true upgrade to the formula.
__________________
|
04-25-2016, 03:40 AM | #23 | |
[Nobody liked that.]
Join Date: Sep 2012
Posts: 3,358
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Quote:
"What does this system offer, that skill ranking, and access to in-depth level ranks / token ranks, neglect?" My main point is if you asked me is from a design perspective, what kind of good does this leader board do us? I can't answer it with a better answer than "nothing." Also I see you are not a vet, and I am unsure how new you actually are, so for reference when I say in-depth level ranks: (Also I don't remember if this is a thing given to everyone, please ignore this if it is. It's been a while.) If anything, tier points gave us one thing: A list of songs higher players placed weight on. Which is GREAT knowledge. It's great. Really. That can be used for the new skill rating metric. But beyond that, it's lost it's place. Especially when the skill rating system came in. (I am unsure if this thing references TPs. I think it does, which means it'd need a modification constantly which makes that equation kinda... Not as useful the longer we don't update it.) Really though, I apologize if this seems like i'm digging at you. I can't stress hard enough that's not my intention. You honestly have a hell of a knack for stuff like this if you asked me. Also should note, I could be wrong on something, or entirely. It's just what I think. Last edited by XelNya; 04-25-2016 at 03:41 AM.. |
|
04-25-2016, 03:43 AM | #24 |
FFR Player
Join Date: Dec 2007
Location: nima
Posts: 4,278
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
|
04-25-2016, 03:52 AM | #25 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Haha, no you're not digging at me. And you're right to a certain degree. I've, for the most part, transitioned from using the skill rating formula to update TPs to using the TP requirements as benchmarks to create a more accurate skill rating formula.
(And yes, the "detailed" level ranks are available to all members. At least, I have the levelranks that you display in that screenshot.) That being said, I might...have tinkered with the constants a little :P (or actually I just got rid of the extrapolated data from 92-96 and re-calculated) AAAeq formula itself remains the same a = -0.085482 + (-0.002106+0.085482)/(1 + (DIFF/81.72714)^10.97649) b = 0.0005004*DIFF^2 - 0.08213*DIFF + 4.082412 c = 139273100 + (13.58178-139273100)/(1+(DIFF/452.0376)^8.394673) I really don't think I'm going to mess with this any more...unless another hole shows up that I think I can fix...this is probably as accurate as I can get considering the limitations of the data.
__________________
Last edited by RenegadeLucien; 04-26-2016 at 10:03 PM.. |
04-26-2016, 10:34 PM | #26 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
I had a spare hour and a half, so...Program that will take your levelrank and rate all your songs with this formula, plus give you an overall skill rating
The program (JAR) How to run: paste your levelranks into a text file (only the songs/scores/PGAMB/etc themselves, not the other stuff on the page), and call the file "levelrank.txt" put the text file and the JAR in the same directory run the JAR, you will get your results in a file called "results.txt"
__________________
Last edited by RenegadeLucien; 04-26-2016 at 10:35 PM.. |
04-27-2016, 09:07 AM | #27 |
Digital Dancing!
Join Date: Feb 2006
Location: 80 billion club, NE
Age: 31
Posts: 12,980
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
A few points that I'll respond to:
1) Tier points are unnecessary I pretty much agree with this. I believe TPs have served their purpose and have no real meaning anymore. I think that tiers should be replaced by Divisions which are extrapolated from the Skill Rating algorithm. That being said, I think we can still use them to sharpen the algorithm, like Renegade is attempting to do. 2) The current skill rating algorithm is inaccurate I think it does well in the lower and mid spectrum of songs, but on the high-end, it gets a little hairy. This is most likely a product of the skill ramp taking a hike at that level and the algorithm isn't able to capture exactly what the hell is going on up there. (This is anything 85+). 3) Who tf even is this guy he's a total noob and he's walkin around like he owns the place (paraphrase) Definitely not his first account. Out of respect of privacy, I won't spill the beans.
__________________
Last edited by rushyrulz; 04-27-2016 at 09:07 AM.. |
04-27-2016, 10:18 AM | #28 | |
Spun a twirly fruitcake,
Join Date: Feb 2009
Age: 31
Posts: 3,865
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
1000 goods on a dif 90 is the same as AAAing a dif 18
ok then
__________________
Quote:
|
|
04-27-2016, 01:03 PM | #29 | ||
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Quote:
[Though Rushy is right about the not first account thing.] Quote:
__________________
|
||
04-27-2016, 01:38 PM | #30 |
Digital Dancing!
Join Date: Feb 2006
Location: 80 billion club, NE
Age: 31
Posts: 12,980
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
It was a pretty satirical paraphrase, nobody thinks you're a noob walking around like you own the place.
__________________
Last edited by rushyrulz; 04-27-2016 at 01:38 PM.. |
04-27-2016, 02:26 PM | #31 |
Anime Avatars ( ◜◡^)っ✂╰⋃╯
Join Date: Mar 2007
Location: Squat Rack
Age: 34
Posts: 10,837
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
I agree that under the assumption the skill rating formula produces accurate estimations of score value, the tier point list isn't necessary. I was hoping, however, that the list at some point in the future could serve a different purpose. This is something I pitched to multiple devs over the past couple years (and eventually something similar [albeit significantly more complex than what I had in mind] arose in the smleaderboards site)
I was hoping to use the tp list to help develop a player pentagonal stat indicator that would show up somewhere on profiles. For reference, this is what it looks like (I'm sure most players have seen something like this at some point) Here's a snippet from the last time I discussed it (to save myself some time): [2015-09-11 4:57:44 PM] Rob: basically [2015-09-11 4:58:06 PM] Rob: i wanted to base it on the songs used for tier points [2015-09-11 4:58:21 PM] Rob: incorporate an invisible stat attribute towards whatever the song stresses [2015-09-11 4:58:25 PM] Rob: between like [2015-09-11 4:59:22 PM] Rob: speed, stamina, jacking, technicality (bursts etc., maybe there's a better term to represent this) and consistency (for measuring accuracy on longer files...potentially something better for the fifth slot could come along idk, maybe separate speed for streaming and speed for jumpstreaming) [2015-09-11 5:00:12 PM] Rob: we could have a pentagonal stat indicator groove radar thing on player profiles ranging from...idk 1-10? [2015-09-11 5:00:26 PM] Rob: anyways, we'd assign invisible stats to each tier point song [2015-09-11 5:01:13 PM] Rob: a high FGO like eclipse for example worth 10 points on the tp list could be worth 9 points in speed when the player meets max requirements for the highest tier [2015-09-11 5:01:58 PM] Rob: with points awarded for each subsequent tier met underneath max scaling downwards appropriately [2015-09-11 5:02:26 PM] Rob: lots of songs will probably fall into multiple categories worth different amounts [2015-09-11 5:02:48 PM] Rob: like for instance a AAA on something could be worth speed 5 but technicality 8 [2015-09-11 5:03:26 PM] Rob: i'd have to go through everything and assign these numbers, hopefully with the help of other diff consultants or players experienced enough to understand what each file stresses and to what degree [2015-09-11 5:04:10 PM] Rob: i don't know if it should be structured so that a player's rating on the radar is the max point value achieved for that category for any one score [2015-09-11 5:04:20 PM] Rob: or if you'd prefer a weighted average using top four or something [2015-09-11 5:04:22 PM] Rob: a la skill rating [2015-09-11 5:04:35 PM] Rob: but yeah you get the gist i think (i hope) [2015-09-11 5:04:45 PM] Rob: i articulated better in staff chat i wish i could find it aarrghh [2015-09-11 5:06:36 PM] Rob: if using a 1-10 scale for the five categories, i believe anyone achieving a 9 in speed for one score is very capable of achieving multiple 8s [2015-09-11 5:06:41 PM] Rob: assuming we just used 1-10 integers [2015-09-11 5:07:27 PM] Rob: so i think basing player profile score value for each category on just the highest points received on any one file would be sufficient (and accurate) [2015-09-11 5:07:46 PM] Rob: since the gaps between one level and the next would be large enough [2015-09-11 5:08:00 PM] Rob: but if you don't like that i'm sure doing something like 1-100 could work [2015-09-11 5:08:19 PM] Rob: it'd just be a lot more time-intensive to come up with the values for each file [2015-09-11 5:08:24 PM] Rob: (and much more subjective lol) To inevitably answer "why only the tier point songs and not everything?", the tier point files are already segmented in such a way that points for the stat indicator can be assigned based on the tier boundaries, and most of the tier files were picked based on the fact that they stress those categories to high degrees at each difficulty level. Plus it'd be incredibly time-intensive to assign these values for all 2k songs in game and I honestly have no intention on doing that assuming this took off (especially continuing to do so on a rolling basis as new songs come out, it'd be much more efficient to just update the tier list with new files if more appropriate charts to test these skills came out). Aside from players getting to jack off to another statistic (and one with a neato graphic to boot!), I think it'd be cool for players to be able to quantify and visually measure their improvement in any skill category, instead of just guessing they're getting better at some specific skill over time. [2015-09-11 5:22:16 PM] Rob: plus i could see random people profile surfing looking for really funny stat indicators [2015-09-11 5:22:30 PM] Rob: like something with a 9 in jacking but 5/6s everywhere else [2015-09-11 5:22:54 PM] Rob: a 'post your stat radar' thread could be started at some point shortly after release if it becomes a thing Plus yeah penis graphs are always neat to see Last edited by One Winged Angel; 04-27-2016 at 02:31 PM.. |
04-27-2016, 02:33 PM | #32 |
Keepin it Real since '05
Join Date: Mar 2005
Location: Steamboat Springa, CO
Age: 34
Posts: 549
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
i like that pentagonal stat thing. how much work would that actually entail
__________________
|
04-27-2016, 02:42 PM | #33 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Well one thing's for sure, it would require the help of D6-7 players that could tell what songs needed what skillsets. But unless I'm understanding it incorrectly, the actual implementation wouldn't be all that difficult. Just might be somewhat tedious, because it would be pick song-> analyze song for skillset required-> put numbers in a table-> repeat 200 times.
__________________
|
04-27-2016, 02:59 PM | #34 |
FFR Player
Join Date: Dec 2007
Location: nima
Posts: 4,278
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
it would take 3 people under 4 hours to do relatively accurate arbitrary ratings for a player skillset radar for all of ffrs tier'd files
assuming they dont spend all their time arguing with each other they'd also actually have to do it thats the real challenge Last edited by MinaciousGrace; 04-27-2016 at 03:00 PM.. |
04-27-2016, 03:01 PM | #35 |
Anime Avatars ( ◜◡^)っ✂╰⋃╯
Join Date: Mar 2007
Location: Squat Rack
Age: 34
Posts: 10,837
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
and whenever that happens I'd probably just steamroll over them with whatever my values were
|
04-27-2016, 03:05 PM | #36 |
FFR Player
Join Date: Dec 2007
Location: nima
Posts: 4,278
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
steamroll more like beanrice
|
04-27-2016, 03:48 PM | #37 |
Mostly Ignored
Skill Rating Designer
Join Date: Mar 2011
Posts: 471
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
So, this concept of the difficulty of a file coming from specific characteristics would a great precursor to more advanced stuff, so based on that I'd love to see it done.
__________________
2014 October 7th 1:03 AM Zageron: Trumpet Trumpet63: yes, im here Zageron: You have a problem. |
04-30-2016, 08:29 PM | #38 |
Fractals!
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
so...a GROOVE RADAR??
|
05-1-2016, 12:42 AM | #39 |
FFR Veteran
Skill Rating Designer
Join Date: Jan 2016
Age: 27
Posts: 282
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
What "more advanced stuff" are you referring to here?
__________________
|
05-1-2016, 04:02 AM | #40 | |
🥓<strong><span style="col
Resident Overseer
Join Date: Mar 2007
Location: Kingsport, TN
Posts: 7,648
|
Re: Suggestion/Proposal: Tier Point Requirements Revamp
Quote:
__________________
|
|
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|