Queue/Batch Discussion Thread v2

[Wyde]

Originally Posted by kommisar

oh nuts i get to judge two of my songs lmao. hi wyde

'sup

Yeah funny you got landed with that lol. hope u liek

[qqwref]

I'm not sure if I agree that previous judges should judge the resubmission files again. On one hand, it would ensure that any fixes that got pointed out actually get fixed; on the other hand, if one person thinks the song or chart style is total crud, they are not going to improve their opinion much when they see that a few minor problems have been changed. I guess you can put me down for a tentative yes, but I'd still like to see the worst grade thrown out in cases of extreme subjectivemania (+ + + -, ++ + + ?, that kind of stuff).

[TC_Halogen]

Throwing out grades is a bad idea in the fact that a file might be objectively incorrect, but will have passing grades for multiple judges anyway for various reasons (judging fatigue, extreme bias toward song choice when chart is passable but not worthy), etc.

tbh, Arch's ratings were a bit all over the place. Hell, even I thought +. was a bit ridiculous for my chart Cheeseburger last batch -- that file shouldn't have gotten anything higher than a +?, lol

If the song has a poor rating, it should only be considered thrown away if the judge can't give legitimate reasons as to why. Obviously if a judge gives a bad grade, they've most likely done it for a reason and if they can prove it, then their grade shouldn't be thrown out because it's rude to them for judging it.

Take me for example in my last batch, and Altair (which I believe I gave it a [-]). What if it received good marks from all of the other judges? My notes get thrown away for no reason?

It's just a thought; it's not a bad idea, but it comes with a lot of issues behind it.

As for resubmissions: pass them around to different judges, because there might be an inevitable bias towards/against songs.

[buizel8888]

I think we should all go play o2jam. Just sayin

[qqwref]

Originally Posted by TC_Halogen

Throwing out grades is a bad idea in the fact that a file might be objectively incorrect, but will have passing grades for multiple judges anyway for various reasons (judging fatigue, extreme bias toward song choice when chart is passable but not worthy), etc.

It's possible, but I think it's a lot more likely that one person messes up than that three people mess up. Anyway I don't think the notes should always be thrown out, or the judge booted, just that the lowest rating shouldn't be counted when we determine how well songs did.

Originally Posted by TC_Halogen

If the song has a poor rating, it should only be considered thrown away if the judge can't give legitimate reasons as to why. Obviously if a judge gives a bad grade, they've most likely done it for a reason and if they can prove it, then their grade shouldn't be thrown out because it's rude to them for judging it.

This is a good idea - to ask the lowest judge (if that rating is the lowest by a lot) to explain in more detail why they gave that ranking and what could be fixed. Then we could throw out their result only if they can't give a reasonable explanation.

The thing which makes me want to throw out the lowest rating is that the current system can potentially give a judge too much power. If one person gives a file [?] or worse, there is nothing the other judges can do - it's basically getting flat out rejected. The purpose of team judging is to give each file multiple perspectives, but it's important to have a system where a judge can't personally prevent a file from getting ingame.

[leonid]

Divide the judging team into two groups (instead of three), and for each file, omit the highest and the lowest score

[ichliebekase]

Originally Posted by qqwref

On one hand, it would ensure that any fixes that got pointed out actually get fixed; on the other hand, if one person thinks the song or chart style is total crud, they are not going to improve their opinion much when they see that a few minor problems have been changed. I guess you can put me down for a tentative yes, but I'd still like to see the worst grade thrown out in cases of extreme subjectivemania (+ + + -, ++ + + ?, that kind of stuff).

You pretty much said it for me in this, but

Originally Posted by TC_Halogen

If the song has a poor rating, it should only be considered thrown away if the judge can't give legitimate reasons as to why. Obviously if a judge gives a bad grade, they've most likely done it for a reason and if they can prove it, then their grade shouldn't be thrown out because it's rude to them for judging it.

This should be considered too. If the judge cannot give good solid reasons as to why it received a lower rating, then it really shouldn't be considered. Pretty much means that they didn't do their job [in the sense that they should be pointing out mistakes or flaws].

Put me down for a yes like qqwref.

[Arch0wl]

Originally Posted by TC_Halogen

Throwing out grades is a bad idea in the fact that a file might be objectively incorrect, but will have passing grades for multiple judges anyway for various reasons (judging fatigue, extreme bias toward song choice when chart is passable but not worthy), etc.

tbh, Arch's ratings were a bit all over the place. Hell, even I thought +. was a bit ridiculous for my chart Cheeseburger last batch -- that file shouldn't have gotten anything higher than a +?, lol

Does it surprise you that someone whose position is that simfiles cannot be objectively evaluated would not have an objective method of judging? It is objective with respect to my charting style, but that's as far as anyone can go. A method is not objective just because it is systematic.

[Patashu]

I would have +'d cheeseburger if I was judging it. idg why you hate it. I guess the <3 of exotic swing is lost when you transfer it into ffr though

[Niala]

Originally Posted by leonid

Divide the judging team into two groups (instead of three), and for each file, omit the highest and the lowest score

+100 for this.

[jimerax]

increasing judge's load by a lot = no

[TC_Halogen]

Originally Posted by Arch0wl

Does it surprise you that someone whose position is that simfiles cannot be objectively evaluated would not have an objective method of judging? It is objective with respect to my charting style, but that's as far as anyone can go. A method is not objective just because it is systematic.

I guess I can say it does surprise me a little bit. Truthfully, there's nothing really erroneous about the chart rhythm wise, but the overall structure is a bit off.

Originally Posted by Patashu

I would have +'d cheeseburger if I was judging it. idg why you hate it. I guess the <3 of exotic swing is lost when you transfer it into ffr though

It really is lost...you can AAA the entire song just using the 96th swing as 16th runs; kinda sad really.

Backstory: The chart was actually rushed for a tournament that happened on FFR, as was Strikebeam, AND The Crow's Newspaper (although TCN was the first chart I worked on; as a result, it came out the best out of the three). I created the final rounds for every division and put them on the DF engine, and then pulled them off once the tournament was over. I don't hate my chart, it's just not properly structured -- more freeform than anything. But hey, it's cool to see that you liked it

[kommisar]

maybe people should just deal with judgments and fix their files


the judges haven't been selected to spite your files; they know how to make good files and will tell you how to do it.

[bmah]

So far, that's what the consensus seems to be. The answer is a no, then. Keep making suggestions though.

[who_cares973]

are we sending batch notes to the email again or to bmah?

[kommisar]

I've always sent them to bmah ._.

[ichliebekase]

Yea, the judges don't get picked to spite files, they just get picked in some random order as it seems.

Maybe the teams should be consistent? Unless this created a problem in the past that I don't know about.

[who_cares973]

keeping people on their toes. what judge you gunna get this batch?! who knos

[who_cares973]

i stepped virt(oh shit) and more ronald jenkees(oh shit) along with nero went to legoland(oh shit)

next batch is going to be a big heavy hitter batch for me(all files unreleased too!!)

[woker-X]

too bad that the normal batch will be closed some time