Serious Judging System Suggestion

[i love you]

Well I know one thing...

I'm glad that the judging system is being reconstructed lol

[Xiz]

Originally Posted by i love you

Well I know one thing...

I'm glad that the judging system is being reconstructed lol

[yo man im awesome]

Originally Posted by Izzy

I didn't read anyone's suggestions, but his comment was just so self righteous it hurt.

what

[reuben_tate]

Originally Posted by Xiz

Hallelujah

[TC_Halogen]

Originally Posted by i love you

Well I know one thing...

I'm glad that the judging system is being reconstructed lol

Yeah...

There were really a lot of miscommunications, and we're aiming to fix that so a massive mishap doesn't happen again; it's going to take some time, but it's being actively worked on. I'm starting to sound like a broken record, but everyone just needs to be patient while things get completely kinked out a bit. Everything requires time.

EDIT: ok seriously I need to stop coming here and do my fucking homework lol

[Xiz]

Originally Posted by TC_Halogen

Everything requires time.

Just like a good quiche! You don't wanna overcook the eggs, otherwise you will be eating a sponge! It requires tender love and care.

[Rebirth0]

While I'd also love to debate with you on some of your own responses, I'll back off after this. Don't want to piss off the staff again and get banned for no good reason again. (That's meant to be humorous while at the same time, a hopeful wake up call. Your attitude is just like Tass's back in the day. I highly suggest not becoming him.)

Maybe I needed to explain in a very detailed fashion everything I said before, because you claim to be picking apart my arguments while not understanding what they're supposed to make you think about

the judges group didn't falter because of it being too many files, they faltered because of miscommunications, and literally because of certain people's inability to just motivate themselves to do it.

I said I thought people were overworked. What do you think inability to motivate themselves is? I see it as "wow I have to judge 100 files, that's so many, I don't want to do that" and put it off forever. You can't tell me none of the judges felt that way.

implying that you know better than others about what makes a good file and what doesn't.

Knowing "better than others about what makes a good file and what doesn't" is exactly what a judge is. They're picked to do exactly that. It's something I did a long time ago and I can still pick out what is good and what isn't. I don't doubt many people have improved over time, and that's a good thing, but I also don't need to actively submit to the batch to know what I'm talking about.

If the game were to be purged or have yearly scoreboards/ranked files, and the list was growing, that's completely different.

Of course that would be great. With the community deciding what files to purge, or eventually make a different client/database with something like only 2 - 3 files of each difficulty (on a 100 point scale) go into that game. But yes I know that won't happen, at least the purging, because hurr synth likes the full history of the game to be displayed


If judging positions become available again, I'll apply or whatever I have to do. I feel like doing this stuff again.

[Izzy]

Originally Posted by Rebirth0

Knowing "better than others about what makes a good file and what doesn't" is exactly what a judge is. They're picked to do exactly that. It's something I did a long time ago and I can still pick out what is good and what isn't.

It is still incredibly subjective. I'm sure for the most part nearly anyone who has played the game significantly can determine what a good file is when they see it. What needs to happen is the community as a whole should form some method of objectifying the specifications of what kind of things are good or bad in a simfile.

So maybe with time comes experience with this kind of thing, but that doesn't help the process as a whole. There should be a more concrete basis between judges.

If that was accomplished than you wouldn't really need a select list of judges who are somehow determined to be better at picking a good file than others.

[cedolad]

An approval process similar to osu!'s would be a practical approach.

[ilikexd]

Originally Posted by Rebirth0

because the game is getting too massive

for whom? What are you judging this by?

Originally Posted by Rebirth0

there are still pretty weak files getting into the game

Care to say which ones, AND explain why? I would say that the quality has been increasing as people have been getting better and more experienced.

Originally Posted by Rebirth0

your judges shat themselves, my guess because there are too many files to judge.

Yes, some did. And some went the extra mile. The problem isn't with the judges or system as some whole, it's that there was a coinciding of numerous personal failures and the fact that a lot of people had to put stuff off for final exams; a few people were behind, which motivated others to not worry about being on time because they thought they wouldn't be the last ones, and a couple just straight out did nothing -- it's their personal fault and it's being dealt with... it's not the system or the workload, as can be demonstrated by the fact that other judges did everything well and within the timeframe.

Originally Posted by Rebirth0

And if you really think you want to let people submit 5+ files per batch, then enforce stricter judging. Right now if I was judging a 70 file batch, I would tell myself "I AM ONLY ACCEPTING X AMOUNT OF FILES." There will be deserving files that would be left out, but that's why you're called a judge. Make the decision to pick the absolute best files.

Again, this would be a decent point if there was some need to control the total volume of charts we'd want in the game. But I don't agree with that. And either way, it would lead to marginalizing stepartists who weren't as elite but were still capable of producing charts slightly above acceptance line that are still enjoyable to the community, and would really eliminate the opportunity for new stepartists to rise up.

Originally Posted by i love you

Well I know one thing...

I'm glad that the judging system is being reconstructed lol

"lol"

That smugness is really appealing to no one as it's known that you were a significant cause of the troubles with last batch.

Originally Posted by Rebirth0

Knowing "better than others about what makes a good file and what doesn't" is exactly what a judge is. They're picked to do exactly that. It's something I did a long time ago and I can still pick out what is good and what isn't. I don't doubt many people have improved over time, and that's a good thing

Originally Posted by Rebirth0

but I also don't need to actively submit to the batch to know what I'm talking about.

You might.

Originally Posted by Rebirth0

If judging positions become available again, I'll apply or whatever I have to do. I feel like doing this stuff again.

Please do.

[ilikexd]

Originally Posted by Izzy

It is still incredibly subjective. I'm sure for the most part nearly anyone who has played the game significantly can determine what a good file is when they see it.

This is demonstrably so far from the truth. What happens a lot is people putting down (but also praising, mind you) charts just based on their own personal experiences, frustrations, and, at times, whimsical opinions. It's especially bad when they have no experiencing making charts. It really takes a lot of experience and a good mindset to be able to judge charts.

Originally Posted by Izzy

What needs to happen is the community as a whole should form some method of objectifying the specifications of what kind of things are good or bad in a simfile.

Although actually something like this would be really cool to see.

[Izzy]

Originally Posted by cedolad

An approval process similar to osu!'s would be a practical approach.

Which is what? I'm not familiar with this game.

[Rebirth0]

Originally Posted by ilikexd

Care to say which ones, AND explain why? I would say that the quality has been increasing as people have been getting better and more experienced.

If you seriously want me to, I could list some files. I haven't played a ton of the newest files, but I know some of them felt dumb. I could spend some time later to go over a lot of the files from the last year. It'll have to be later today or something since I'm about to leave

Yes, some did. And some went the extra mile. The problem isn't with the judges or system as some whole, it's that there was a coinciding of numerous personal failures and the fact that a lot of people had to put stuff off for final exams; a few people were behind, which motivated others to not worry about being on time because they thought they wouldn't be the last ones, and a couple just straight out did nothing -- it's their personal fault and it's being dealt with... it's not the system or the workload, as can be demonstrated by the fact that other judges did everything well and within the timeframe.

I know some of the judges did the job like they should, and it is the personal failings of certain judges that blew this up. But it is also true that lowering the amount of submissions makes it even less likely for that to happen.

[i love you]

Originally Posted by ilikexd

"lol"

That smugness is really appealing to no one as it's known that you were a significant cause of the troubles with last batch.

Just the miscommunication part and the part of me losing my notes and not saying anything about it until two weeks later. Not the batch as a whole. Every judge but one submitted their notes late. I just so happened to be called out for something I did wrong on my part.

Also, I had a good laugh at your "smugness" comment.

[cedolad]

Originally Posted by Izzy

Which is what? I'm not familiar with this game.

http://osu.ppy.sh/wiki/Beatmap_Approval

I'll try my best to explain how it would work for FFR. Since we don't have a client-based game and instead flash/browser based, it's harder for us to do this method. My idea was to have all submissions uploaded to an "Approval Engine", where the community goes and play them; giving notes, suggestions, comments etc. Then when a chart has enough of whatever (thumbs up's, stars or whatever), it's sent to a judge where s/he then approves it for ranking.

It gives the community a chance to have a say in what they find acceptable, and includes them in the process. It also takes a huge work load off of the judging crew, because by the time they see a chart, it will have gone through numerous amounts of changes and all they would need to do is fine polish it and approve it for ranking.

[Izzy]

Yea, that sounds like a great idea to me. The community really needs to be a bigger part of what is good or bad and not the judges.

It would be cool if you could upload any of your own simfiles to your profile and have people play them outside of ranking. If there were enough votes or plays on a simfile there then maybe it could just get auto pushed to ranking.

Honestly anything to remove judges from the equation seems like a good idea to me.

[cedolad]

Of course revamping the current system to this system would require a lot of work. The approval process in itself would require a revamp on the main site as well. Such as a "pending charts" page, where it would be similar to say the old Simfile DB, or the Video DB, with a star ranking and comments section.

[ItsOnlyDanO]

The osu idea would be good, but a problem could occur that some newer players may not understand what charts are good or bad, thus affecting ratings. This could be fixed by only allowing experienced players to rate the charts however.

[Charu]

Originally Posted by ItsOnlyDanO

The osu idea would be good, but a problem could occur that some newer players may not understand what charts are good or bad, thus affecting ratings. This could be fixed by only allowing experienced players to rate the charts however.

This is deja-vu...

Originally Posted by Charu

I still think we should do a system where someone submits, puts it up somewhere so Judges can take a look at it at their own pace and have some non-judge feedback as well.

idk tho

Originally Posted by Xiz

^ going off of your idea Mr. Snivy

Here is a random new judging system idea:

- Have your full judging team of 12 or whatever
- They can judge files whenever they want at their own pace
- Every time a file is submitted it is put in the list and available for download via cloud for judges only.
- A file needs 4(ish) judges to be passed / accepted or whatever. The files might need to be judged in order of submission?


File Name Here (4/4) - 8/7/7.5/5
Other File Here (4/4) - 6/5/5/5
Here is another file (3/4)
One more file (3/4)
Files lol (2/4)
Yet Another Fiiiile (0/4)
Really crappy file by Fission (0/4)


*(#/4) represents judges who have judged the file so far

- Final ratings will be posted in OP only after all ratings are in
- There could be two different groups, but I'd say no more then that.
- That way, you can have the lazy judges judge whenever they feel like it, and the judges who actually do their job keep on trucking like the badasses they are



(Tell me why this wont work, or why it would. I'm actually curious to know how badly this sucks, or how it might be something to work off of lol)


TL;DR: Chart by chart judging system

[cedolad]

Originally Posted by Izzy

It would be cool if you could upload any of your own simfiles to your profile and have people play them outside of ranking. If there were enough votes or plays on a simfile there then maybe it could just get auto pushed to ranking.

Having the ability to upload your own file to the "Approval Engine" would be a really great idea, unfortunately I think that would take a ton of work and huge overhaul would be in order to do something like this.

Honestly anything to remove judges from the equation seems like a good idea to me.

The implication of the new system wouldn't necessarily disband the judging crew. It would just, in a sense, lessen their work load. Instead of playing through tons of files looking for technical errors, they can instead spend time subjectively looking at a file for playability, and it's a replay value. And that's only if a file has met certain criteria that the community as a whole has set.