Go Back   Flash Flash Revolution > Flash Flash Revolution > FFR General Talk

Reply
 
Thread Tools Display Modes
Old 10-1-2014, 07:54 PM   #41
MyNameIsNothing
ʕ◡ᴥ◡✿ʔ
FFR Veteran
 
MyNameIsNothing's Avatar
 
Join Date: Mar 2013
Age: 26
Posts: 519
Default Re: New Skill Analysis Metric (Trial Phase)

64 is a pretty great difficulty.
thanks for the graphs, cool stats.
MyNameIsNothing is offline   Reply With Quote
Old 10-1-2014, 08:35 PM   #42
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

Quote:
Originally Posted by Silvuh View Post
Rawr, haappy to help~ And, yeah, it was kind of funny that with my scaled score, my best difficulty goes down but my score goes up. And it's due to the effect you mentioned: I only have one 80 and one 82 on the chart, 10 goods and a boo on Ketsarku Mozgalom and 12 goods on Blue Rose. My other scores in that range are definitely not comparable.

So rather than divide by the number of files for a difficulty in your sheet, I tried dividing by the total number of files in that difficulty. And this seems to work better; the curve is doesn't fan out so much. (It's equivalent to multiplying the scaled score by the ratio of files you have NGC<20 on, because the count of NGC<20 files cancels out.) My new Z-Score is 67.58 with a best difficulty of 61.

The change on my spreadsheet is pretty simple, and I can update my file if you like the idea.

Sheet2 Column C currently references Sheet1 Column L, which restates the difficulty for played files with a non-negative NGC (new good count). So instead, you can put =IF(B2=0,0,B2/COUNTIF(Sheet1!B:B,"="&A2)) in C2, referring to the complete difficulty list in Column B, and fill that down.


Oh, and the only step in your explanation I got stuck on was Step 2, because it doesn't explicitly say to multiply the Averages, Misses, and Boos by those factors and sum them all up with the number of goods to make a new good count. (And Step 4 still refers to what I'm calling the NGC as the "good count".)
Aww man, you beat me to it, lol. I had been thinking the same thing. Let me think on this, because I'm not yet entirely convinced that this conveys the same concept as the straight average, and that might still be useful. Part of me is concerned that this will very much reduce scores for people who have only played a few songs. In principle, if a player is selecting songs randomly (evenly distributed), then this scaling down perfectly counteracts the difficulty concentration issue, but I find that it reduces my score a bit too much for me to be completely convinced yet. I'd say, keep both versions of the file
And thank you again for the updated formula to put in! (You're a wizard with Excel, I swear...)

Last edited by Trumpet63; 10-1-2014 at 08:37 PM..
Trumpet63 is offline   Reply With Quote
Old 10-1-2014, 09:25 PM   #43
choof
Banned
Retired StaffFFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
Join Date: Nov 2013
Posts: 8,563
Default Re: New Skill Analysis Metric (Trial Phase)

my graph will look like shit
choof is offline   Reply With Quote
Old 10-1-2014, 10:16 PM   #44
Silvuh
quit
Retired StaffFFR Simfile AuthorFFR Music ProducerFFR Veteran
 
Silvuh's Avatar
 
Join Date: Apr 2005
Location: anywhere but here
Posts: 938
Default Re: New Skill Analysis Metric (Trial Phase)

Yeah, I didn't even think about accounting for unplayed songs, because I've played everything except for the most recent stuff (and sometimes I ignore long/hard files for a while.) Trying [TeRa]'s level ranks, because he's in D7 and has a lot of unplayed files. (His best difficulty is 81 in all of these.)

divided by NGC < 20 files:

Z-Score: 85.01 (This chart is similar to the others, where the earlier difficulties have a straight line of all AAAs, starting to scatter around the mid-70s and curving down in the 80s.)

divided by all files:

Z-Score: 84.62 (This chart is scattered throughout with an obvious break after 83, the following scores being much lower.)

So next, I thought to try dividing by all played files. There's still trouble if a person has, like, a good score on the only 80 they've ever played (though I'd think that must be pretty rare and not worth figuring out how to account for until it comes up, and they'd have to deliberately play one they know they can score well on for their skill in that range to be overestimated).

C2=IF(B2=0,0,B2/COUNTIFS(Sheet1!B:B,"="&A2,Sheet1!K:K,">0"))

Z-Score: 84.25 (The graph has a few early dips but is mostly linear at the start and doesn't get as scattered, with a noticeable break at 84.)

Also, sometimes people don't play a file the whole way through. Like, [TeRa] has 540-1-0-21-1 on Daybreak, making a dip in the 44s. The most reliable way to fix this would be including a total note count for each file, but that's not in the level ranks. Some people miss or boo out, but I think some also do that with auto-fail. One way to get some of them out would be checking for "goods < 50 & averages = 0 & (miss >= 20 || boo >=25)" I know for me, at least, that I always get at least one average and a lot of goods on a file I truly miss that much on, but the unfinished scores usually don't.

I didn't know how to do it all in one forumla, so I had to add Sheet1!O2=AND(F2<50,G2=0,OR(H2>=20,I2>=25)) and fill that down, and then Sheet2!C2 becomes =IF(B2=0,0,B2/COUNTIFS(Sheet1!B:B,"="&A2,Sheet1!K:K,">0",Sheet1!O:O,"=FALSE")).

Z-Score: 84.29 (Slightly higher than the previous one. The graph raises a few points a bit, fixing some of the gaps.)

And my new Z-Score with this last one is 68.21, which is closer to the original, non-scaled score. If someone knows they don't have any incomplete plays, they can just use the formula dividing by all played files, because this check for incomplete plays isn't perfect.
__________________
Silvuh is offline   Reply With Quote
Old 10-1-2014, 10:37 PM   #45
TC_Halogen
Rhythm game specialist.
Retired StaffFFR Simfile AuthorFFR Music ProducerD8 Godly KeysmasherFFR Veteran
 
TC_Halogen's Avatar
 
Join Date: Feb 2008
Location: Bel Air, Maryland
Age: 32
Posts: 19,376
Send a message via AIM to TC_Halogen Send a message via Skype™ to TC_Halogen
Default Re: New Skill Analysis Metric (Trial Phase)

I'm interested in seeing my output, haha.
TC_Halogen is offline   Reply With Quote
Old 10-2-2014, 08:54 PM   #46
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

That's everyone!
EDIT: No, actually it's not


Z-Score = 71.06
Z-Score Scaled = 77.45


Z-Score = 74.65
Z-Score Scaled = 92.45


Z-Score = 69.62
Z-Score Scaled = 90.00


Z-Score = 72.82
Z-Score Scaled = 85.44


Z-Score = 86.24
Z-Score Scaled = 85.42


Z-Score = 77.15
Z-Score Scaled = 79.45


Z-Score = 74.37
Z-Score Scaled = 88.55


Z-Score = 74.50
Z-Score Scaled = 91.75


Z-Score = 64.86
Z-Score Scaled = 66.73


Z-Score = 77.12
Z-Score Scaled = 75.03

Last edited by Trumpet63; 10-2-2014 at 09:17 PM..
Trumpet63 is offline   Reply With Quote
Old 10-2-2014, 09:29 PM   #47
One Winged Angel
Anime Avatars ( ◜◡^)っ✂╰⋃╯
Retired StaffFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
One Winged Angel's Avatar
 
Join Date: Mar 2007
Location: Squat Rack
Age: 34
Posts: 10,837
Default Re: New Skill Analysis Metric (Trial Phase)

Very neat.

Just noticed one thing after looking at a few of the curves -- partial and failed runs on files are being taken in account. Like AlphaWolf for instance, his 265-2-0-1-1 on La Camp is being valued the same as 2-0-1-1 on the full file, leading to his scaled Z-score of 90.
__________________


Quote:
Originally Posted by ilikexd View Post
i want to be cucked by cirno
One Winged Angel is offline   Reply With Quote
Old 10-2-2014, 10:00 PM   #48
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

Quote:
Originally Posted by One Winged Angel View Post
Very neat.

Just noticed one thing after looking at a few of the curves -- partial and failed runs on files are being taken in account. Like AlphaWolf for instance, his 265-2-0-1-1 on La Camp is being valued the same as 2-0-1-1 on the full file, leading to his scaled Z-score of 90.
Yep, that's a "known issue". thank you for pointing out this instance of it, because it will probably be used in analysis. I or Silvuh will grab a list of note totals in the future and make a change in the formula to remove incomplete scores.
Trumpet63 is offline   Reply With Quote
Old 10-2-2014, 10:11 PM   #49
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

Alright! I'm caught up


Z-Score = 68.92
Z-Score Scaled = 82.27


Z-Score = 71.61
Z-Score Scaled = 74.44


Z-Score = 34.28
Z-Score Scaled = 48.66


Z-Score = 88.93
Z-Score Scaled = 87.00


Z-Score = 74.61
Z-Score Scaled = 84.93


Z-Score = 78.05
Z-Score Scaled = 84.16


Z-Score = 52.72
Z-Score Scaled = 60.39


Z-Score = 68.35
Z-Score Scaled = 70.21


Z-Score = 43.08
Z-Score Scaled = 55.90

Last edited by Trumpet63; 10-2-2014 at 10:11 PM..
Trumpet63 is offline   Reply With Quote
Old 10-2-2014, 10:15 PM   #50
Poison-
Nope
FFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
Poison-'s Avatar
 
Join Date: Nov 2010
Location: London, ON
Age: 29
Posts: 3,772
Default Re: New Skill Analysis Metric (Trial Phase)

Apparently I'm almost as good as Kevin's Weeney

Last edited by Poison-; 10-2-2014 at 10:16 PM..
Poison- is offline   Reply With Quote
Old 10-2-2014, 10:23 PM   #51
~Zero~
FFR Veteran
FFR Simfile AuthorFFR Veteran
 
~Zero~'s Avatar
 
Join Date: Nov 2006
Age: 32
Posts: 1,556
Send a message via AIM to ~Zero~ Send a message via Skype™ to ~Zero~
Default Re: New Skill Analysis Metric (Trial Phase)

I would like to see mine as well. Please and thank you.
__________________


Quote:
Originally Posted by Xiaounlimited View Post
More like Tier ~0~
~Zero~ is offline   Reply With Quote
Old 10-2-2014, 10:39 PM   #52
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

Here ya go!

Z-Score = 70.14
Z-Score Scaled = 85.68

Last edited by Trumpet63; 10-2-2014 at 10:39 PM..
Trumpet63 is offline   Reply With Quote
Old 10-2-2014, 10:49 PM   #53
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

Okay, let's start some analysis! Silvuh, if you want to try adding another column to this with one of your formulas, be my guest. I think I'm done for tonight.

Last edited by Trumpet63; 10-2-2014 at 11:16 PM..
Trumpet63 is offline   Reply With Quote
Old 10-2-2014, 10:55 PM   #54
remedy1502
remederpin
FFR Veteran
 
remedy1502's Avatar
 
Join Date: Feb 2007
Location: Ontario, Canada
Age: 31
Posts: 4,884
Default Re: New Skill Analysis Metric (Trial Phase)

What's the difference between the Zenith score and scaled?

lol this table
remedy1502 is offline   Reply With Quote
Old 10-2-2014, 11:00 PM   #55
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

Quote:
Originally Posted by remedy1502 View Post
What's the difference between the Zenith score and scaled?
It is a difference in how the calculated weights are used. The standard Zenith Score takes the sum of the weights within a difficulty, while the scaled takes the average. These values are seen in the y-axis. Visit the OP for a better explanation of the calculated weights.

More specifically, the scaled represents your average good count among songs that you have less than 20 goods in (with higher values representing a better score).

Last edited by Trumpet63; 10-2-2014 at 11:03 PM..
Trumpet63 is offline   Reply With Quote
Old 10-2-2014, 11:05 PM   #56
remedy1502
remederpin
FFR Veteran
 
remedy1502's Avatar
 
Join Date: Feb 2007
Location: Ontario, Canada
Age: 31
Posts: 4,884
Default Re: New Skill Analysis Metric (Trial Phase)

Quote:
Originally Posted by Trumpet63 View Post
It is a difference in how the calculated weights are used. The standard Zenith Score takes the sum of the weights within a difficulty, while the scaled takes the average. These values are seen in the y-axis. Visit the OP for a better explanation of the calculated weights.
Word. Also my stuff if you want to use it in the table you just posted above:

Z-score: 70.9897914
Scaled: 79.3884094
D6
remedy1502 is offline   Reply With Quote
Old 10-2-2014, 11:51 PM   #57
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

I guess I'm not done tonight. Here's some minor analysis on the table of players. It seems the scaled version is better than I thought!

Trumpet63 is offline   Reply With Quote
Old 10-3-2014, 12:02 AM   #58
Kanzas
Buh
FFR Veteran
 
Kanzas's Avatar
 
Join Date: Jul 2014
Posts: 1,459
Default Re: New Skill Analysis Metric (Trial Phase)

Very interesting.
__________________




Top 5
Pimp Slap - 7-0-0-1
party 4u "holy nite mix" v2 - 5-0-1-2
Fei Longer - AAA
HELLBEAT - 9-0-1-3
Flame Repellent - 5-0-0-0
Kanzas is offline   Reply With Quote
Old 10-3-2014, 12:31 AM   #59
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trial Phase)

Quote:
Originally Posted by Untimely Friction View Post
TIL I'm d6 by 0.19... value... on the scaled zenith curb 8)
To be fair, the scales are based on the sample, so it's more like you're 0.19 from the lowest person in D6 for whom we have data. Oh, and you're referencing the 74.25 "Combined by Average" number? That is an amalgamation of the regular and the scaled, and it's not on the table. So, you'd have to compare your 74.44 with the 76.12. (I used the word average on the table to mean something else. Sorry it was misleading)

Also, Zenith Score is in units of difficulty... so, you are now 1.68... difficulties... away?

I should add, if you're curious you can compare yourself to others in your division (who are not listed here) using the link in the OP, if you have Excel.
Trumpet63 is offline   Reply With Quote
Old 10-3-2014, 02:54 AM   #60
Silvuh
quit
Retired StaffFFR Simfile AuthorFFR Music ProducerFFR Veteran
 
Silvuh's Avatar
 
Join Date: Apr 2005
Location: anywhere but here
Posts: 938
Default Re: New Skill Analysis Metric (Trial Phase)

Yeah, I touched on the incomplete play issue earlier but didn't want to collect all the notecounts at the time. But I finally got this. (I started working at like 10:30, haha, and now I see I missed a few posts in here, welp!) Anyway, download link is the same.

zenith-score.xlsx

Writing a script to collect all the FFR file notecounts was much easier than trying to figure out how to get Excel's alphabetical sort to match FFR's, which ... I wasn't able to do, haha.

So there's a few more steps than before now:

1. Copy table data from levelrank.php
2. Sheet1!A1 should already be selected, so paste your data into the spreadsheet.
3. The pasted data will be selected, so you can go to Sort & Filter > Custom Sort
4. Sort by Level on Values with order A to Z so the note counts match up properly.
5. Enter Sheet2 to find your results.

The scaled score now divides by all completely played files of that difficulty. This does mean I'll have to update the spreadsheet every time new songs are uploaded, but that won't be a problem for me to do.

EDIT:

Oh, one thing you can add to your table is just a column of like (if the Google sheet uses the same notation as Excel)

=IF(ISBLANK(D2)=FALSE,D2=IF(C2<20,"D1",IF(C2<40,"D2",IF(C2<50,"D3",IF(C2<60,"D4",IF(C2<70,"D5",IF(C2<80,"D6","D7")))))),"")

filled down. First it checks if a user has an Actual Division Placement, outputs nothing if false, then assigns the scaled score to a division and checks if it's the same as the ADP, outputting TRUE or FALSE. Maybe you already did something like that for your table in post 57 though.

Oh, and just to say again, some version of the scaled score will definitely be more accurate, because the score is dependent on one's Best Difficulty. Like, a D7 player's Best Difficulty should not be 64, so the original score underestimates their skill.
__________________

Last edited by Silvuh; 10-3-2014 at 03:25 AM..
Silvuh is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -5. The time now is 08:30 AM.


Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright FlashFlashRevolution