Batch judging suggestion

[who_cares973]

Re: Batch judging suggestion

The problem is finding those qualified to judge

[rushyrulz]

Re: Batch judging suggestion

cross post from batch discussion thread in march

I do think that subjectivity plays a bigger role in the judging process than just the fun factor. If a file is technically correct with halfway decent PR there is no way in hell it should get a difference between the highest and lowest grades, greater than 2. I can see someone getting 5/x/3/x, but 5/x/2/x?? Come on, something isn't right here. 5/5/1/5 is out of the question, One judge shouldn't have the power to TRUMP reject all the other judges in the team, that's just insane.

When I see these score breakdowns, it sometimes leads me to believe that either some of the judges are unqualified to judge, or unable to judge with objectivity. Sure different judges have different opinions on the 'no-no' level of some sections of files, but I sometimes think that giving a rating of 2 solely because of a slight sync problem is a tad overboard. You've trump rejected a file because someone didn't hit F10/F11 enough.

So my honest opinion? Yes this system needs an overwork. Since we only have 12 judge slots, I don't think doing 2 teams would be the best option because of the overload of files to judge. 4 judge teams of 3 would give each judge even more trump power than they have currently. The two alternatives I have in mind would be:

a. Get more judges

This might not be a possibility seeing as how it's hard enough as is to get everyone to finish on time. (see: Patashu) Although I will admit, the current judge team is doing a really good job of getting notes done on time.

The more probable alternative would be:

b. Toss out the oddball score and make an accept 12 or 13/15

4 judges will still be in a team, just the rating that is most different from the others will be tossed out, resulting in a 15 point system. For example:

kommisar works really hard and makes the most badass file FFR has ever seen. The batch notes come out and he gets [++] [+] [+.] [?]. Under the current judge system, this file would not get queued because it only totals to 16/20. BUT if you toss out the odd 2/5 score, the total would be 14/15; A passing score. A different example:

Dr4g0nSly40r347 throws some piece of hindu throat singing crap together in half an hour. The notes come out and he gets [-] [?] [?] [+.]. While this would be rejected under both systems, the odd +. would be thrown out making for a more accurate depiction of how much of a piece of crap the file actually was. (9/20 vs. 5/15)




TL;DR: Current system gives too much power to the individual judge, we should throw out the most odd score to make a 'majority' type decision rather than a unanimous one.

[ScarletSky]

Re: Batch judging suggestion

Problem I see with that idea then becomes: why have that extra judge there to begin with. I'm sure he/she doesn't want to feel like just because they felt strongly (justified or not) about a file, that they're opinion doesn't matter anymore just because of that.

Edit~
Plus, it sounds eerily similar to the OP's post. But instead of singling out 2 judges, it's just one.

[who_cares973]

Re: Batch judging suggestion

That shouldn't matter. The review wouldn't deleted or ignored. People would need to get over themselves if this is the case. I think most of the judges would be on board with the new judging system that jx created

[ScarletSky]

Re: Batch judging suggestion

I like the new system JX is trying to implement myself, but I do agree that for it to work efficiently we need more qualified judges to help as well.

[who_cares973]

Re: Batch judging suggestion

Too bad there are no more

[hi19hi19]

Re: Batch judging suggestion

Put Rave7 in the game and I will judge a batch