View Single Post
Old 11-17-2014, 07:33 PM   #9074
Silvuh
quit
Retired StaffFFR Simfile AuthorFFR Music ProducerFFR Veteran
 
Silvuh's Avatar
 
Join Date: Apr 2005
Location: anywhere but here
Posts: 938
Default Re: Queue/Batch Discussion Thread v2 [Regular batch: OPEN]

There have been discussions on here about the pros and cons of different systems, and ... I don't really remember the arguments, but it does seem like the batch system is just what works for us. "Works" relatively better, anyway.

Like, when we realized that some of the May/June files had been skipped over, I uploaded them and posted in the forum and the chat about getting extra judgments, and after a while, only one other judge had stepped up to look at them.

And you all saw how it was over a month between posting May/June's initial and final results because of the passed files. When there's a group of files that no one's assigned to look at and there's no schedule, things just don't get done. And I feel like it'd just keep being like that if we tried a "judge when you want to" system.


Yeah ... No one's happy that it took this long to start judging the July/August stuff. The original date for completing May/June judgments was the last day of August. Then that September happened ...

I opened signups for judging the July/August batch around the end of September. I didn't expect to start judging right away, because the May/June stuff still wasn't totally settled. But I also didn't expect it to take six weeks for eight judges to sign up. And not every judge offered to look at two sets, so we don't actually have every set covered right now. I released two sets for judging on Friday so each judge would be looking at one set.

And now you're probably thinking, "Why didn't you just release half the batch the moment eight slots were filled?" And ... I honestly have no idea, and I'm hitting myself about that now. I don't know. It's not something we've done before that I remember, so I just wan't thinking about it. I'm sorry. I should have thought of doing that sooner. But we'll be doing that from now on.


On another note ... as part of minimizing passed files, we've started collecting hard files into fewer sets, so judges who prefer not to judge hard files can be assigned sets without them.

We're also going to start doing weekly check-ins now. This should help with accountability.

We've also been working on a more robust judge application process. We know we need more help, but we need to make sure that help is qualified.
__________________
Silvuh is offline