Queue/Batch Discussion Thread v2

[Silvuh]

Because of the limited activity between batch judgments, I did end up putting Nightfall (Original Mix) and They Kidnapped the Princess (and the other four files that had been skipped over) in with the July/August batch. Sorry for not mentioning that in here earlier.

[XelNya]

Originally Posted by Silvuh

Because of the limited activity between batch judgments, I did end up putting Nightfall (Original Mix) and They Kidnapped the Princess (and the other four files that had been skipped over) in with the July/August batch. Sorry for not mentioning that in here earlier.

I'm happy to hear it Thanks

[M0nkeyz]

Originally Posted by Silvuh

Because of the limited activity between batch judgments, I did end up putting Nightfall (Original Mix) and They Kidnapped the Princess (and the other four files that had been skipped over) in with the July/August batch. Sorry for not mentioning that in here earlier.

is that batch being judged atm? or hasn't judging started yet?

[SKG_Scintill]

so quiet

[choof]

Go ahead and pull odd22, having the judges play it would just be a waste of time.

[DossarLX ODI]

For the record: Judging for the July/August Batch started yesterday.

[Gradiant]

At the risk of sounding like a dick:
Could we maybe get a different sort of system in place so that we're not just now starting getting ratings for an almost 3 month old batch?

[Charu]

Originally Posted by Gradiant

At the risk of sounding like a dick:
Could we maybe get a different sort of system in place so that we're not just now starting getting ratings for an almost 3 month old batch?

Osu system~~~~~~~~~~~~~~~

[African Anger]

add african anger on osu! thanks guys

[bmah]

Originally Posted by Gradiant

At the risk of sounding like a dick:
Could we maybe get a different sort of system in place so that we're not just now starting getting ratings for an almost 3 month old batch?

Although I'm also complaining about the long wait, please consider that there was a transition from handing the game management from AJ to Silvuh and Dossar. A rocky start certainly affected the efficiency of the system; it is not the infrastructure of the system itself that's the issue. Recall that a lot of changes have already occurred to the judging infrastructure over the past year - not to mention a few more additions that Silvuh has implemented. It'll take some time, but I hope we can catch up quickly.

[Charu]

Well if you guys need any help, y'know, wink wink, squirm squirm, CHARU.

[FFR Pro 21]

Originally Posted by OP

-Nov15-
Hajnal v2 (FFR Pro 21)

This should say "Hajnal 2 (FFR Pro 21)".

[Gradiant]

Originally Posted by bmah

Although I'm also complaining about the long wait, please consider that there was a transition from handing the game management from AJ to Silvuh and Dossar. A rocky start certainly affected the efficiency of the system; it is not the infrastructure of the system itself that's the issue. Recall that a lot of changes have already occurred to the judging infrastructure over the past year - not to mention a few more additions that Silvuh has implemented. It'll take some time, but I hope we can catch up quickly.

Understandable, but theres always several people willing to help out shown by charus post, yet rather than ask for a bit of help to keep up during/after transition, it falls behind and people say 'oops sorry' and pick back up again just to fall behind again. It seems to happen this way every couple batches and itd just be great to have it picked back up and stay that way rather than beng 2 batches late every other batch.

[Silvuh]

There have been discussions on here about the pros and cons of different systems, and ... I don't really remember the arguments, but it does seem like the batch system is just what works for us. "Works" relatively better, anyway.

Like, when we realized that some of the May/June files had been skipped over, I uploaded them and posted in the forum and the chat about getting extra judgments, and after a while, only one other judge had stepped up to look at them.

And you all saw how it was over a month between posting May/June's initial and final results because of the passed files. When there's a group of files that no one's assigned to look at and there's no schedule, things just don't get done. And I feel like it'd just keep being like that if we tried a "judge when you want to" system.


Yeah ... No one's happy that it took this long to start judging the July/August stuff. The original date for completing May/June judgments was the last day of August. Then that September happened ...

I opened signups for judging the July/August batch around the end of September. I didn't expect to start judging right away, because the May/June stuff still wasn't totally settled. But I also didn't expect it to take six weeks for eight judges to sign up. And not every judge offered to look at two sets, so we don't actually have every set covered right now. I released two sets for judging on Friday so each judge would be looking at one set.

And now you're probably thinking, "Why didn't you just release half the batch the moment eight slots were filled?" And ... I honestly have no idea, and I'm hitting myself about that now. I don't know. It's not something we've done before that I remember, so I just wan't thinking about it. I'm sorry. I should have thought of doing that sooner. But we'll be doing that from now on.


On another note ... as part of minimizing passed files, we've started collecting hard files into fewer sets, so judges who prefer not to judge hard files can be assigned sets without them.

We're also going to start doing weekly check-ins now. This should help with accountability.

We've also been working on a more robust judge application process. We know we need more help, but we need to make sure that help is qualified.

[Charu]

How to be qualified?

[M0nkeyz]

well halogen promises that the batch will be on schedule then we don't hear from you for ages and now we are 2 months behind again =/

[bmah]

As I said already, we're under new management, so to speak. That was on AJ's terms, and since he's stepping down, that guarantee is no longer there. I'd like for the schedule to be on track once Silvuh/Dossar's management settles in a bit.

[VisD]

Anyway it's not like the files are going anywhere fast. There are files in the queue which have been there for like over a year

edit: Hopefully this batch is at least completed by the end of the year though, so re-submissions and fixes can be in time for Nov/Dec batch. And new judge applications soon plz ty

[DossarLX ODI]

It would also be appropriate for me to mention that I'm working towards some automation in the judging process to reduce manual work. Silvuh and I are working towards fast generation of an excel spreadsheet for adjusted ratings that is not only scalable to a batch of any size, but also can be created with scripts instead of writing stuff out by hand which takes a long time.

In the future batches will consistently be done in smaller sets when there are enough judges available as opposed to waiting for many judges for an entire batch -- judge availability is very time sensitive and the delay here can be seen as a wake-up call.

[Wayward Vagabond]

Close the batch until we catch up then reopen and start fresh