[WirryWoo]

Originally Posted by Gradiant

How so? A player wouldn't really need to play any new files after their top 50/100/whatever were filled, unless they felt the new file would be a promising one to add to their top scores. The only people anything would be asked of really would be newer players, which boils back down to the minimum number of files played being a basic requirement.

See below:

Originally Posted by WirryWoo

Instead, let's look at past data where we know for a fact that songs are released in some sequential order. Before 2004, when files are held in Legacy engine, there is a good amount of opportunities to score well on songs requiring trilling: One Minute Waltz, Flight of the Bumblebee, and Runny Mornings (SGX Mix), debatably Molto Vivace. Players who excel at trilling will perform relatively well on these files and will highly benefit from any system imposed. You can consider their performance as "flukes" (similar to Zag's performance on Club and AIM Anthem). Due to smaller file frequency in the engine, a well designed skill rating system back then would require something like Top 10, regardless if is weighted or unweighted.

Fast forward to today, we get La Campanella, Giselle, MAX Forever, and other trilly songs that I can't think of right off the top of my head. But I'm very confident that there are at least 10 songs currently in the engine that emphasizes trilling. This is equivalent to Zageron having more files similar to Club and AIM Anthem, and therefore, has more opportunities to fluke. Top 10 will easily be filled with trilly songs, so there is a need to scale out. This time span is less than 20 years at least for trilling files. For other patterns, this length will vary depending on previously mentioned variables (stepartist song submissions, batches, events, etc.).

Regardless of how long it takes for the need of scalability to exist, the main point to make about scalability is that there is a point in time somewhere in the future where we need to revamp the system. Maybe Top 100 would be too small of a sample size, so we need Top 150, or Top 200, etc. in order to maintain the accuracy of skill ratings.

The issue posed with the unweighted setting is that it will be difficult to retrack these players who joined FFR at 2002 and then stopped playing the game. Your solution (which I personally characterize as "hacky") is to filter out these players so that their ratings don't get factored into the overall high scores. This goes back to my thoughts about the minimum requirement:

When you filter these players out, this also changes the definition of "skill rating". Imagine if Usain Bolt did not participate in Summer Olympics this year but attended four years ago. Has his skill rating changed? Maybe, maybe not. Point is, he still should relatively have the skills to perform at the Olympics-level if he were to attend. Your suggestion is to mark him as "no skill due to not participating", where my solution respects his skills given his past performance and attempts to make that acknowledged despite not seeing his performance this summer. Which one better measures skill?