Go Back   Flash Flash Revolution > Flash Flash Revolution > FFR General Talk

Reply
 
Thread Tools Display Modes
Old 09-30-2014, 08:52 PM   #1
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default The Zenith Score Project Thread



Status: Dormant
See Also: All Current Recorded Zenith Scores

zenith-score.xlsx (v.2.1.2)
If you have Excel, you can use this. Go to your level ranks page and copy the whole table, including the column titles (in Firefox, it's some kind of Ctrl+Shift with clicking and dragging from just outside the table). Paste it into B1 of Sheet1, go into Sheet2, and Sheet3, and all the results will be there. (Credit to Silvuh)

Alternatively, you can post in the thread and I will return the results to you, after which I can use your scores to improve the formula.

The Zenith Score is a measure of skill potential in units of difficulty.

This is the most recent scale by average:



Pretentious name is pretentious, I know. _Zenith_ was the first person other than myself that I used this on, and I thought it sounded cool. The graph above will be called the Zenith Curve, and the number you get from it will be called the Zenith Score. Zenith also had no idea I was posting this.

So far I'm only using Excel to do the calculations because I do not have a working knowledge of any programming language that can work with chunks of data such as this. In the end it would really only create issues with implementation if this became widely used.

Any feedback would be tremendously helpful. Having an opportunity to share this with people, and to get people excited about the enormity of the statistics here would make me very happy.

And just to give you a taste, the graph above indeed belongs to Zenith, and his Zenith Score is 72.01.

Volunteer Appreciation Here:
Tim Allen
PhantomPuppy
noname219
LethalMutiny
Callipygian
================

How the Original Works: (10 steps)
1. I copy the levelranks table (of course) and paste it into Excel.

2. Each score is sorted by the Good Count, and a value is assigned to other non-goods in the following way:
Perfect = 0
Good = 1
Average = 1.8
Miss = 2.4
Boo = 0.2
(Each of the P/G/A/M/B counts are multiplied by these values, then added)

3. Every score with 20 goods or above is removed from the table.

4. Each score is then assigned a new value, based on the following formula, hereafter referred to as the Weight.
Weight = (20-[Good Count])*[Difficulty]

5. Next, the weights for each difficulty are added together separately and arranged in a new table in the following way:
Column1 = "Difficulty" || Column2 = "Sum of Weights"

6. (At this point the graph could be made) The actual Zenith Score is obtained by first isolating all of the weights associated with the difficulty AFTER the highest weight value, or the "peak". i.e. All the values to the right of the peak.

7. Take the sum of all the values in the Sum of Weights column that have been isolated, this will be called Sum1.

8. Make another column that is the product of Difficulty and Sum of Weights:
Column3 = "DW Product" = [Difficulty]*[Sum of Weights]

9. Take the sum of all the values in this new DW Product column, this is will be Sum2 (again, only the ones to the right of the peak).

10. Zenith Score = Sum2 / Sum1


News and Progress:

9/30/14 I considered changing the Sum of Weights to an Average of Weights (the Y-value). Doing this was an effort to account for the fact that the difficulties in the game are not evenly distributed (refer to chart below). However, I found that when the sample for a particular difficulty is very small, the weight will become artificially high (i.e. one song played at D70 will imply you have AAA's on all D70 songs).



10/1/14 Thanks to the help of Silvuh, I have been able to conduct many more trials in much less time. After re-testing my Zenith Score, I found it to be 39.95, which, according to the current model of the scale, puts me borderline D2/D3. Since I was one of the last to be placed in the Tenth Official, and was placed into D3, but was previously D2, it is evident to me that I am in fact borderline D2/D3. Based on this and mr_candy and Zenth's scores agreeing with their current division placement as well, I am still hesitant to say the original formula is terribly flawed.

I postulate that a change in the division scale could be a more effective solution than developing a new calculation method. Nevertheless, I and Silvuh agree that an intuitive solution to the difficulty concentration problem would be to divide each weight by the number of songs in each difficulty. Yet more testing is necessary to rule out the usefulness of either method, and indeed I still find that the Average of Weights method conveys something with the graph that is still very powerful. For today I have to leave this issue of the three methods alone because of time constraints. I'd like to add that I don't like the idea of punishing players for playing in a difficulty range that has a lot of songs, or in effect making those scores less significant, and this is what this new counter-weight method seems to do.

10/2/14 A significant sample of 30 players has been assembled into a table to examine the viability of the Zenith Score and its variations for division placement. Adjustments to the scale should come next. In addition, two changes are to be made: 1) The length of the spreadsheet needs to be enlarged to account for the song additions caused by the Tenth Official. 2) A complete list of note counts for public songs needs to be assembled and integrated into the spreadsheet. It is likely that the note counts have to be arranged in alphabetical order because that would be the only arrangement that is consistent among all players.

10/5/14 Note counts added, sheet extended, and non-fully-played songs discounted thanks to the Excel work by Silvuh. We are adding a couple new dimensions to the Zenith Score by including new versions of the scaled score in a attempt to grasp at the concept of skill, and try to accurately predict division placements. Also the original scaled score is receiving an overhaul to correct the one-score-in-a-difficulty problem. After the spreadsheet is fully updated, we will conduct all of the trials on the table again, and afterward move into profiling entire divisions' scores. We may be looking for volunteers to conduct this large number of trials.

10/12/14 After many bugfixes in the spreadsheet, thanks to Silvuh, we can finally move into the next phase of development by making a complete survey of the tournament and its participants. Hopefully this will help us determine which of the six methods currently in use are best, and help us to gain a deeper understanding of the statistics themselves.

10/13/14 The entire tournament has been surveyed, thanks to several volunteers, and the work of Silvuh and myself. In addition, the larger portion of analysis as been completed. We have found the average, standard deviation, maximum, and minimum in each division, and for each flavor of Zenith Score. The next task is finding out the ideal scale of scores that results in the lowest error count. Current error estimations are at about 19% for W4, over all the divisions. After determining the best method to make the division scales, and finalizing the error count, the next step will be to hypothesize changes to the formula that may reduce the error count.

Last edited by Trumpet63; 11-13-2014 at 09:01 AM..
Trumpet63 is offline   Reply With Quote
Old 09-30-2014, 10:00 PM   #2
mr_candy
SPEEEEEEEEEEEEEEEEEEEEEED
FFR Veteran
 
mr_candy's Avatar
 
Join Date: Jul 2008
Posts: 196
Default Re: New Skill Analysis Metric (Trail Phase)

tl:dr do me do me
__________________
Quote:
Originally Posted by hi19hi19 View Post
Dossar reached critical mass after the 9th Official and ascended to a being of pure Stepmanian energy.
You won't be seeing him in any more competitions as he has already joined his life force with the arrows.

In our hearts, every competitor is Dossar now. that or I'm guessing he's busy in school and doesn't want distractions.
mr_candy is offline   Reply With Quote
Old 09-30-2014, 10:04 PM   #3
Sorie
FFR Player
 
Sorie's Avatar
 
Join Date: Jul 2014
Age: 35
Posts: 142
Default Re: New Skill Analysis Metric (Trail Phase)

I'd like to see my Zenith Score/Curve. I'm expecting a descending curve.

That curve..

Last edited by Sorie; 10-1-2014 at 09:14 AM..
Sorie is offline   Reply With Quote
Old 09-30-2014, 10:08 PM   #4
One Winged Angel
Anime Avatars ( ◜◡^)っ✂╰⋃╯
Retired StaffFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
One Winged Angel's Avatar
 
Join Date: Mar 2007
Location: Squat Rack
Age: 34
Posts: 10,837
Default Re: New Skill Analysis Metric (Trail Phase)

why is there a sudden influx of these types of threads lol

sure I guess haha
__________________


Quote:
Originally Posted by ilikexd View Post
i want to be cucked by cirno
One Winged Angel is offline   Reply With Quote
Old 09-30-2014, 10:11 PM   #5
PrawnSkunk
Administrator
User Administrator
AdministratorDeveloperFFR Simfile AuthorFFR Veteran
 
PrawnSkunk's Avatar
 
Join Date: Dec 2007
Location: Vancouver, BC
Age: 29
Posts: 3,902
Default Re: New Skill Analysis Metric (Trail Phase)

yes, but what about the Zenith Zenith Zenith Zenith Zenith
PrawnSkunk is offline   Reply With Quote
Old 09-30-2014, 10:24 PM   #6
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trail Phase)



Your Zenith Score is: 59.82
Your best difficulty is: 45
With 3 songs under 20 goods, and an average of 2.07 goods among them

EDIT: Fixed? with updated formula

Last edited by Trumpet63; 09-30-2014 at 11:08 PM..
Trumpet63 is offline   Reply With Quote
Old 09-30-2014, 10:30 PM   #7
mr_candy
SPEEEEEEEEEEEEEEEEEEEEEED
FFR Veteran
 
mr_candy's Avatar
 
Join Date: Jul 2008
Posts: 196
Default Re: New Skill Analysis Metric (Trail Phase)

Quote:
Originally Posted by Trumpet63 View Post


Your Zenith Score is: 66.14
Your best difficulty is: 60
With 13 songs under 20 goods, and an average of 8.48 goods among them
Cool....what does that mean?
__________________
Quote:
Originally Posted by hi19hi19 View Post
Dossar reached critical mass after the 9th Official and ascended to a being of pure Stepmanian energy.
You won't be seeing him in any more competitions as he has already joined his life force with the arrows.

In our hearts, every competitor is Dossar now. that or I'm guessing he's busy in school and doesn't want distractions.
mr_candy is offline   Reply With Quote
Old 09-30-2014, 10:36 PM   #8
MyNameIsNothing
ʕ◡ᴥ◡✿ʔ
FFR Veteran
 
MyNameIsNothing's Avatar
 
Join Date: Mar 2013
Age: 26
Posts: 519
Default Re: New Skill Analysis Metric (Trail Phase)

seems cool, i'd like to see mine.
MyNameIsNothing is offline   Reply With Quote
Old 09-30-2014, 10:38 PM   #9
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trail Phase)

Quote:
Originally Posted by mr_candy View Post
Cool....what does that mean?
Essentially, if your skill could be described by a difficulty level, 66.14 would be it... so far... if my system is valid at all.
It only uses songs for which you have less than 20 goods, keep in mind.
Trumpet63 is offline   Reply With Quote
Old 09-30-2014, 10:39 PM   #10
CammyGoesRawr
nocturnal girl (〜✘﹏✘)〜
FFR Veteran
 
CammyGoesRawr's Avatar
 
Join Date: Jul 2011
Location: Pennsylvania
Age: 30
Posts: 2,477
Default Re: New Skill Analysis Metric (Trail Phase)

do it for me~
__________________

CammyGoesRawr is offline   Reply With Quote
Old 09-30-2014, 10:41 PM   #11
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trail Phase)



Your Zenith Score is: 76.57
Your best difficulty is 68
With 24 songs under 20 goods, and an average of 0.6 among them.

EDIT: Fixed with the formula changes

Last edited by Trumpet63; 09-30-2014 at 10:59 PM..
Trumpet63 is offline   Reply With Quote
Old 09-30-2014, 10:51 PM   #12
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trail Phase)

Actually... Sorie, your scores made me rethink something.
I think instead of using the sums of the weights for each difficulty, I need to use the average because the number of songs in each difficulty vary wildly, and you've AAA'd like all of the songs below 62 (wow really?).

Recalculating your Zenith Score, I get 76.57.

I'll redo yours mr_candy, and I'll change the graph for both of you to see if it works better. (I may just edit the post to prevent bumping)

ALSO: I'm tired guys, and I have to study for my Biology test tomorrow. Let me revisit this in a day or so, after I make the above changes to the two people I did already.

Last edited by Trumpet63; 09-30-2014 at 10:52 PM..
Trumpet63 is offline   Reply With Quote
Old 09-30-2014, 10:53 PM   #13
mr_candy
SPEEEEEEEEEEEEEEEEEEEEEED
FFR Veteran
 
mr_candy's Avatar
 
Join Date: Jul 2008
Posts: 196
Default Re: New Skill Analysis Metric (Trail Phase)

yeah no rush mate
__________________
Quote:
Originally Posted by hi19hi19 View Post
Dossar reached critical mass after the 9th Official and ascended to a being of pure Stepmanian energy.
You won't be seeing him in any more competitions as he has already joined his life force with the arrows.

In our hearts, every competitor is Dossar now. that or I'm guessing he's busy in school and doesn't want distractions.
mr_candy is offline   Reply With Quote
Old 09-30-2014, 10:59 PM   #14
blindreper1179
Vice President Of TGB
FFR Veteran
 
blindreper1179's Avatar
 
Join Date: Jun 2006
Location: repping tha D!
Age: 35
Posts: 5,891
Send a message via AIM to blindreper1179 Send a message via Skype™ to blindreper1179
Default Re: New Skill Analysis Metric (Trail Phase)

If you've got the time, I'd like to see my graph. No rush.
__________________
Quote:
Originally Posted by thesunfan View Post
absolutely I want to vomit on your face irl
Quote:
Originally Posted by choof View Post
It was like trying to throw logic at a fuckin brick wall lmao
Quote:
Originally Posted by choof View Post
whats more dense, a black hole or an icyworld file
Quote:
Originally Posted by Celirra View Post
I've never been so disappointed by a man from Alabama than I am right now
blindreper1179 is offline   Reply With Quote
Old 09-30-2014, 11:15 PM   #15
Trumpet63
Mostly Ignored
Skill Rating Designer
Retired StaffFFR Veteran
 
Trumpet63's Avatar
 
Join Date: Mar 2011
Posts: 471
Default Re: New Skill Analysis Metric (Trail Phase)

See, I love this... although the averaging does eliminate the problems with song-difficulty concentration, it makes it very easy to get misleading information from a particular difficulty where a player may have played just a few songs but got very good scores.

I think the previous version of the system will be more valid, especially if I somehow added a weight to each difficulty that counteracts the difficulty concentration issue. I'll start working on that first... tomorrow... before mr_candy freaks out about how I put him in D4.

EDIT: THANK YOU for your patience

Last edited by Trumpet63; 09-30-2014 at 11:16 PM..
Trumpet63 is offline   Reply With Quote
Old 09-30-2014, 11:25 PM   #16
TheSaxRunner05
The Doctor
FFR Veteran
 
TheSaxRunner05's Avatar
 
Join Date: Apr 2006
Age: 35
Posts: 6,145
Default Re: New Skill Analysis Metric (Trail Phase)

Test me when you get a chance, I've played almost every song multiple times so I should be a good sample
__________________


TheSaxRunner05 is offline   Reply With Quote
Old 09-30-2014, 11:27 PM   #17
mr_candy
SPEEEEEEEEEEEEEEEEEEEEEED
FFR Veteran
 
mr_candy's Avatar
 
Join Date: Jul 2008
Posts: 196
Default Re: New Skill Analysis Metric (Trail Phase)

Quote:
Originally Posted by Trumpet63 View Post
See, I love this... although the averaging does eliminate the problems with song-difficulty concentration, it makes it very easy to get misleading information from a particular difficulty where a player may have played just a few songs but got very good scores.

I think the previous version of the system will be more valid, especially if I somehow added a weight to each difficulty that counteracts the difficulty concentration issue. I'll start working on that first... tomorrow... before mr_candy freaks out about how I put him in D4.

EDIT: THANK YOU for your patience


na i'm d4 bro, halogen look i'm d4. he has evidence
__________________
Quote:
Originally Posted by hi19hi19 View Post
Dossar reached critical mass after the 9th Official and ascended to a being of pure Stepmanian energy.
You won't be seeing him in any more competitions as he has already joined his life force with the arrows.

In our hearts, every competitor is Dossar now. that or I'm guessing he's busy in school and doesn't want distractions.
mr_candy is offline   Reply With Quote
Old 09-30-2014, 11:42 PM   #18
_Zenith_
Accuracy Player
Retired StaffD8 Godly KeysmasherFFR Veteran
 
_Zenith_'s Avatar
 
Join Date: Mar 2011
Location: Connecticut
Age: 28
Posts: 4,628
Default Re: New Skill Analysis Metric (Trail Phase)

Never in my life have I seen so many of me @_@

Anyways, you can Go Advanced if you edit your OP and fix the spelling error if it's bothering you for the title.

Also if you want to do me again and update since I played a shit ton of songs tonight, be my guest
__________________




_Zenith_ is offline   Reply With Quote
Old 10-1-2014, 01:59 AM   #19
Lambdadelta
D7 Elite Keymasher
D7 Elite KeysmasherFFR Veteran
 
Lambdadelta's Avatar
 
Join Date: Oct 2012
Age: 30
Posts: 1,149
Default Re: New Skill Analysis Metric (Trail Phase)

me too
__________________
Lambdadelta is offline   Reply With Quote
Old 10-1-2014, 04:04 AM   #20
shadow 1800
I can't think of anything
D7 Elite KeysmasherFFR Veteran
 
shadow 1800's Avatar
 
Join Date: Dec 2006
Location: Colorado Springs
Age: 30
Posts: 1,183
Send a message via Skype™ to shadow 1800
Default Re: New Skill Analysis Metric (Trail Phase)

i'd like to see mine when you get around to it, this sounds neat
__________________
shadow 1800 is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -5. The time now is 07:57 PM.


Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright FlashFlashRevolution