Re: Queue/Batch Discussion Thread v2
 

All 4s and one 1 hah thats funny

This judging is pretty all over the place.

So i got three 4s, and one 1, im assuming that 1 killed my chances of getting in, awesome.

Re: Queue/Batch Discussion Thread v2
 

jx... you forgot to add Serenade of Storms to the queue list.



And for the people wondering about resubmissions... I remember that jx said there's going to be a mini-batch for resubmissions only.

Re: Queue/Batch Discussion Thread v2
 

Inconsistency~manias. Think the judging process needs an overhaul yet? Or at least needs to be looked at to see what is going wrong? A file shouldn't receive +. +. +. - let alone + + + - like Dossar got. If no one sees these types of scores as a problem then there's seriously something wrong.

Re: Queue/Batch Discussion Thread v2
 

The judging process isn't as abysmal as people are making it out to be. Files that have wild ranges can be attributed to pushing extreme concepts (look at Metro and BCND). If a file has been submitted multiple times and doesn't go anywhere because it keeps landing the same ratings by multiple judges, then there may in fact be an underlying problem (or problems) within it.

Judge opinions sway towards different concepts.

Take Illumination, where it was 4/4/5/2 (me being the 2) -- I thought the song was problematic because it played out in a very repetitive nature, even for the length...which is < 1:00 -- if a file is diverse and "boring", then it's incredibly subjective. If it's repetitive and "boring", then it might indicate a problem. This concept in itself is subjective, but at least it's logical.

There's multiple files in this batch in different groups that do that, too:

The Bird's Concrete Nosedive: 2/1/2/5+ (10+)
Corrupted Minds: 3/1/5/5 (14)
(another of which was my doing, but I have specific reasons, too) He's a Pirate - F-777 Remix: 4/2/5/4 (15)
Kirlian Changes: 3/4/1/5 (13)

Different aspects of objectivity are getting attacked, and subjectivity is playing a role with some judgments. With this kind of judging setup, there's not much that can be done about it. You have to be consistently good.

Re: Queue/Batch Discussion Thread v2
 

The file can still be sent in though it's just put in as a new submission rather than a resubmit.

Re: Queue/Batch Discussion Thread v2
 

Not really consistently good - more like consistently innocuous. I'll explain. Keep in mind that a 1-2 is not just a bad rating but a REALLY bad one, which is almost certain to kill the file. The current system has a possibility of judges that will give a song a really bad rating basically because they subjectively don't like the style/difficulty/whatever, rather than because the song is objectively really bad.

So what happens is that you can get good files - by which I mean synced, technically sound, without really awkward patterns - that get demolished by the judging process. I'm really surprised Metro made it in this time, not because I think it's a bad file, but because it's just so controversial. Doing something different well seems to come with a very significant chance of some very bad grades, which means the current setup is biased towards people who step things you just can't objectively complain about. Perfectly synced/PR'd easy files, standardized rock files with no really hard parts... those will have an easy time getting in, but if you try something different, you run a substantial risk of one judge misinterpreting what you're trying to do (or just not agreeing with it) and personally rejecting the file.

Re: Queue/Batch Discussion Thread v2
 

We should really base teams on how the judges judge.

Re: Queue/Batch Discussion Thread v2
 

You wish.

+. +. +. +. is 16/20, which isn't queued (17/20).

Re: Queue/Batch Discussion Thread v2
 

Nice experiment Frankenstein 8)

Re: Queue/Batch Discussion Thread v2
 

wow that's retarded as shit if your serious:-|

Re: Queue/Batch Discussion Thread v2
 

He is. Sad, isn't it. If you get a 16 though it goes into a vote and all of the judges vote if they think your file is good enough or not. Usually, it's either yes, no, or yes with fixes. If you do get a 16 though, they'll let you know what the vote came to.

Re: Queue/Batch Discussion Thread v2
 

wow the judging system is highly flawed if that's the case. all [+.]'s should be a pass i mean seriously how is it not? It should be a the lowest possible accepted grade.

Re: Queue/Batch Discussion Thread v2
 

<3

:twisted:

:twisted:

Re: Queue/Batch Discussion Thread v2
 

I do think that subjectivity plays a bigger role in the judging process than just the fun factor. If a file is technically correct with halfway decent PR there is no way in hell it should get a difference between the highest and lowest grades, greater than 2. I can see someone getting 5/x/3/x, but 5/x/2/x?? Come on, something isn't right here. 5/5/1/5 is out of the question, One judge shouldn't have the power to TRUMP reject all the other judges in the team, that's just insane.

When I see these score breakdowns, it sometimes leads me to believe that either some of the judges are unqualified to judge, or unable to judge with objectivity. Sure different judges have different opinions on the 'no-no' level of some sections of files, but I sometimes think that giving a rating of 2 solely because of a slight sync problem is a tad overboard. You've trump rejected a file because someone didn't hit F10/F11 enough.

So my honest opinion? Yes this system needs an overwork. Since we only have 12 judge slots, I don't think doing 2 teams would be the best option because of the overload of files to judge. 4 judge teams of 3 would give each judge even more trump power than they have currently. The two alternatives I have in mind would be:

a. Get more judges

This might not be a possibility seeing as how it's hard enough as is to get everyone to finish on time. (see: Patashu) Although I will admit, the current judge team is doing a really good job of getting notes done on time.

The more probable alternative would be:

b. Toss out the oddball score and make an accept 12 or 13/15

4 judges will still be in a team, just the rating that is most different from the others will be tossed out, resulting in a 15 point system. For example:

kommisar works really hard and makes the most badass file FFR has ever seen. The batch notes come out and he gets [++] [+] [+.] [?]. Under the current judge system, this file would not get queued because it only totals to 16/20. BUT if you toss out the odd 2/5 score, the total would be 14/15; A passing score. A different example:

Dr4g0nSly40r347 throws some piece of hindu throat singing crap together in half an hour. The notes come out and he gets [-] [?] [?] [+.]. While this would be rejected under both systems, the odd +. would be thrown out making for a more accurate depiction of how much of a piece of crap the file actually was. (9/20 vs. 5/15)




TL;DR: Current system gives too much power to the individual judge, we should throw out the most odd score to make a 'majority' type decision rather than a unanimous one.

Re: Queue/Batch Discussion Thread v2
 

I'd like to know if the judge who has't submitted their notes in for Sun and stars can please do that if they are online, because it's weird that only 1 judge's notes is missing for it, and it's the judge who decides the file's fate.

EDIT: I think personally that Rushy's idea about dropping the lowest score of all judge's scores is a beneficial idea. Because it's not fair that a file can have all +'s except for that one ?, or - that completely ****'s the shit out of their chances.

Re: Queue/Batch Discussion Thread v2
 

He'll prolly ? it and trump reject.

Re: Queue/Batch Discussion Thread v2
 

If that's the case I will have lost most if not all respect i have towards the current system used to judge files

:neutral:

Re: Queue/Batch Discussion Thread v2
 

Step a more perfect file that somehow pleases all 4 judges equally the same.

Oh wait that won't even work, because 4(+.) = 16/20 luls

Re: Queue/Batch Discussion Thread v2
 

I'd like to let you all know that my file is actually a borderline file.

[+][+][+?][+?] = 16 not 17.

EDIT: May I still send in a resubmission, or will I have to wait for next batch?