A new beginning for our SM community.

[Reincarnate]

The statistics are just functions of song performance and player performance. Ideally, the new SM client would keep track of this data. Everything scales in real-time and is fully accurate based on available data. You literally can't get a more accurate difficulty rating than this.

[who_cares973]

the problem would be implementing it

[Reincarnate]

why? it's piss-easy to implement

[who_cares973]

nvm i forgot we could use the same system to record scores that smo uses. only problem i see would be distinguishing the multiple files for one song.

[NeoMasterPie]

compare particular data within .sm/dwi files?
edit: to ones already on the sever i would assume

[Reincarnate]

Checksumming is an easy way out of that one (assuming the files are not stored on-server but are played locally and then verified) -- similar to how Flash game data is processed.

[aj187]

The backbone of my ideas for the score/simfile database/online gameplay hinged on hashes of simfile data:

1) Checksum simfile metadata (aside from offset which could be different on each player's computer if they decide to edit files manually? idk)
2) Checksum each chart.

[Reincarnate]

Yeah, hashes are the way to go.

[Oni-Paranoia]

@Reincarnate - Everything you said about the system is exactly what it should be. All though you can't have it be automatic. At this point in time, most people could do more difficult songs relatively easier than some simpler songs for whatever reasons. I don't see the problem with manual rating any songs that come in after the system is in place.

On another note, is the SM for this going to be a build off of 3.9, 4.0, or what... And as for a theme, is there anything in talks? Or is it all still talk of having the site up etc... first?

[ddrxero64]

NPS + FudgeFactor combine an actual system (that will never need to be rerated unless the definition of a second changes or we suddenly evolve into something different than human) with organized subjectivity. To use FudgeFactor on a 1-100 system is to use a subjective system on a subjective system. It makes no sense.

Also, found a huge hole in this thread.

Originally Posted by Shikari

Gellenduo, I said I want to bring the FFR's idea back to the life, but I didn't mention that would be ONLY for keyboard players and stepartists, and people from FFR. Since all this is about an unificated community, of course I'll ask people from OD, Bemanistyle and KBO some help. I consider them separated "categories" because of a multitude of reason: Stepping style, required skills to play, number of players, since we have people who play pad, but not KB; people who play KB, but not pad; people who play OD's files, but aren't aware of keyboard challenging files and pad files, aside of DDR games; people who play solo, and don't play singles anymore. See? They're sorta separated, even being part of the same game.

Which is why, as much as I want to help, it's not my place.

[Reincarnate]

Originally Posted by Oni-Paranoia

@Reincarnate - Everything you said about the system is exactly what it should be. All though you can't have it be automatic. At this point in time, most people could do more difficult songs relatively easier than some simpler songs for whatever reasons. I don't see the problem with manual rating any songs that come in after the system is in place.

On another note, is the SM for this going to be a build off of 3.9, 4.0, or what... And as for a theme, is there anything in talks? Or is it all still talk of having the site up etc... first?

I don't understand what you mean when you say it can't be automatic. I'm operating under the assumption of having difficulties stored external to the stepfile itself. As people played files more and more, difficulty ratings would adjust in real time based on performance. Bayesian weighting will remove initial variance-swing factors as well.

If a "simple song" is still tough to do, then it's that much tougher to do and the weighting will take it into account. The beauty of an objective rating system is that it doesn't matter what people "perceive" a file to be in terms of difficulty.

[Oni-Paranoia]

Originally Posted by Reincarnate

I don't understand what you mean when you say it can't be automatic. I'm operating under the assumption of having difficulties stored external to the stepfile itself. As people played files more and more, difficulty ratings would adjust in real time based on performance. Bayesian weighting will remove initial variance-swing factors as well.

If a "simple song" is still tough to do, then it's that much tougher to do and the weighting will take it into account. The beauty of an objective rating system is that it doesn't matter what people "perceive" a file to be in terms of difficulty.

Automatic was probably the wrong word to use, but yeah. I don't think a files intended difficulty should change for any reason if it's rating is graded correctly in the first place. If a song is a 78, then it's a 78 even if 2 years from now two thousand more people can pass the song, it's still should be a 78.

By the way, let me see if I'm getting this right. 10 beginners fail a song on beginner, song get's rated higher? OML get's AAAed by 50 experts and it's rating get's lowered... The easier song seems harder to pass because no experts play it?

[Reincarnate]

Originally Posted by Oni-Paranoia

Automatic was probably the wrong word to use, but yeah. I don't think a files intended difficulty should change for any reason if it's rating is graded correctly in the first place. If a song is a 78, then it's a 78 even if 2 years from now two thousand more people can pass the song, it's still should be a 78.

By the way, let me see if I'm getting this right. 10 beginners fail a song on beginner, song get's rated higher? OML get's AAAed by 50 experts and it's rating get's lowered... The easier song seems harder to pass because no experts play it?

Ah, I see what you mean. I technically think that difficulty should change with the times. What was considered hard two years ago is quite a lot easier now. If a song is a 78, I think it doesn't mean much for it to stay a 78 even if people start demolishing it -- otherwise you have to start layering on more numbers. I don't like systems where we initially say "10 is the hardest. Wait, make that 11. JK it's 12. Ah hell, this beast is a 22." Past that point it gets too messy and difficulty metrics lose their meaning.

In other words, pretty much this:


Re: Your last point: No, that's what the system is actually meant to guard against. It takes into account who plays the song and how well they do on it. The end result is a market-weighted metric that says "On average, given a completely randomly-drawn player, this is how hard you can expect this file to be, and thusly it will tell you something about what kind of score you should expect for yourself" which is what you want an ideal difficulty ranking to convey.

[Charlo]

I really like this idea. I had just started playing SM as the "glory days" were waning, so something like this is exciting.

[who_cares973]

Thirdstyle difficulty rating

[Reincarnate]

TS doesn't implement it properly

[BeatD0wn]

I'd be up for this! :O

[Bluearrowll]

So just to clarify on what Reincarnate and Oni have stated;

Basically set up a default difficulty 1, and 100, and then for the songs in the middle, set up some form of program algorithm where, based on SMO scores the mean score will dictate what the difficulty is?

Example: Say there's 100 songs played, and AOAO happens to have the 12th lowest raw score. This would mean it is a difficulty 88, because the program will weigh it out vs the other songs.

Would this be difficult to create? It sounds like it would work great if it could be done, but it would have to be embedded in the new client, and it would also make the work that I would be doing (manually changing difficulties) obselete.

[Reincarnate]

There are problems to all sorts of rating systems. Subjective ratings are obviously biased because they may not be accurate and are prone to judgment errors (FFR tries to use a subjective system, and if you actually crunch all the data, you see all sorts of problems where people are dominating, say, a low 7 but sucking on a high 6).

Going by pure performance alone is misleading due to selection bias. Sometimes only sucky players will play a particular song. Sometimes only good players will pick a particular song.

Counting failures in your scores will lead to errors where people are just quitting early because they get bored (or want to play something else) and so on. Best to count passes-only.

This is why you also need to account for *who* is playing your file. So how do you assess who's playing your files? You need a skill metric that objectively defines who owns and who sucks.

So how do you tell who's good and who isn't? % performance is limited as it doesn't tell you WHAT songs you're good at -- so we need to know the difficulty of the files being played.

So you've got a bit of a circular problem. You need to know how hard the files are, but to do this, you need to know how good the players are playing the files -- but to assess how good they are, you need to take into account how hard the files are they've owned, etc.

Which is where Bayesian weighting and/or bootstrapping comes in. NPS would be decent for this. In other words, "load up" each file with a difficulty ranking based on its NPS alone. Now you have something to get a decent estimate for how good a player is (so even if you play only easy files, your performance metric will indicate that you're mainly getting your high % from low-NPS files).

Although NPS would merely be a bootstrap for loading initial values. After that you'd let the skill+performance feedback loop run its course and allow difficulties to shift to their natural places (which ideally wouldn't be far for most files that scale true difficulty from NPS, but perhaps far for files that don't follow the gaussian distribution well).

[who_cares973]

Originally Posted by Reincarnate

TS doesn't implement it properly

Same concept though. Something like this would be pretty sweet. Hopefully all goes well with it