Queue/Batch Discussion Thread

[bmah]

Re: Queue/Batch Discussion Thread

P.S. Regarding files for this upcoming batch, I believe the file limit determined a while ago has not yet been set. In which case, this may be your only chance to send in more than 4 new files. Subsequent batches will have a limit of a maximum of 4 new files per simfiler, excluding resent files.

Also, I know people are already resending files, but from now on, we'd like to ask you to add "[resubmit]" as an extension to the title of song folders, for simfiles that are resubmissions from previous batches.
Since the same judge may not necessarily review the same file you've resubmitted, a list of resubmitted files will help us keep track of files actively undergoing improvement. This should help bring your hard work under more consideration from different judges. It's only fair to do so.

edit: and please, submit in a zip or rar folder...avoid using an external file-hosting site like mediafire rofl
(exception will be made for this batch only)

[i love you]

Re: Queue/Batch Discussion Thread

P.S. Regarding files for this upcoming batch, I believe the file limit determined a while ago has not yet been set. In which case, this may be your only chance to send in more than 4 new files. Subsequent batches will have a limit of a maximum of 4 new files per simfiler, excluding resent files.

Also, I know people are already resending files, but from now on, we'd like to ask you to add "[resubmit]" as an extension to the title of song folders, for simfiles that are resubmissions from previous batches.
Since the same judge may not necessarily review the same file you've resubmitted, a list of resubmitted files will help us keep track of files actively undergoing improvement. This should help bring your hard work under more consideration from different judges. It's only fair to do so.

edit: and please, submit in a zip or rar folder...avoid using an external file-hosting site like mediafire rofl
(exception will be made for this batch only)

I support this. This would make it a lot easier for the judges to keep track of resubmitted files for future batches.

[Coolboyrulez0]

Re: Queue/Batch Discussion Thread

What happens to borderline files, are they played by a "supreme judge" and determined if they are accepted or denied or are they just borderline pass/fail?

Btw 2/4 accepts
1/4 borderline
1/4 reject - due to wc being subjective, and quite frankly being a dick, thats all.

[jimerax]

Re: Queue/Batch Discussion Thread

Just looked at the halogen's list, and yeah acceptance rates among groups are so various.

I'll investigate the judgement strictness of every judge (and quality of every part) and need to determine the final cutoff (since currently 40% acceptance rate, which is considered to be a bit too high).

also there're opinions that psychoangel691 should be backup due to the lack of experience here, so who's considered to be fine with that spot?

ps will set "submission number limit" and "resubmission rule" too.

[dore]

Re: Queue/Batch Discussion Thread

lol I'm gonna resend Elements of a Party. Without Tass it might actually get through

[bmah]

Re: Queue/Batch Discussion Thread

What happens to borderline files, are they played by a "supreme judge" and determined if they are accepted or denied or are they just borderline pass/fail?

I've discussed this with jx not long ago, and to make it simple, we're just going to let numbers decide for us (i.e. refer to the point system) until we've figured out how to add additional variables to address finer points like these. Also, there's no supreme judge parading on the system.

@jx: my first choice would be blu, but the chances are probably slim
next suggestion would be i love you (rebound) for the spot

thoughts?

[jimerax]

Re: Queue/Batch Discussion Thread

Currently we have around 40 submissions (without applying submission cap).

When the number of submissions reaches around 60-70, I'll post batch close annoucement here (batch will be closed 24 hours after that).

[Plan_Bsk81127]

Re: Queue/Batch Discussion Thread

@jx: my first choice would be blu, but the chances are probably slim
next suggestion would be i love you (rebound) for the spot

thoughts?

rebound for sure

[Silvuh]

Re: Queue/Batch Discussion Thread

NOTE: I've been working on this post and another analysis of the point system for about two hours ... I was going to post all the information in one post, but because there have been 12 pots since I've started typing this up... Well, ... I'll just post this part now.

The post:

By JX's suggestions (excluding the accepted + + -) all files with (11) points will be borerline. + +. - is the only borderline (10). So I will associate (15)–(12) as "accepted" and (11) as "borderline".
With everyone fussed on the low acceptance ratio, I will discuss files that may have a higher true rating. My use of "reasonable" refers to the sum altered by the standard deviation.

It's reasonable to assume that + +? ? (10), +. +. ? (10) and + +? - (9) may have a true rating of (11). However, it is unreasonable to assume the same for +. +? +? (10). Out of these three ratings, because (9) is too low, we may could include scores of + +? ? and +. +. ? as borderline.

Increasing the scope of which files are borderline is not a bad idea, because not all of these files will be conditionally queued. Reanalyzing the borderline files will create the "conditionally queued" category and a category corresponding to the old "Played by Tass; not in Queue" category.


But, the same logic can be applied to which files are accepted: if we believe a (12) should be accepted, the borderline scores of + +. ? (11) + +? +? (11) + +. - (10) (three of the current four) could be acceptable. Because we're looking for occurrences of one judge's disagreement effecting a lower score, + +. ? would be the rating most likely for a true acceptance. With + +? +?, the judge's disagreement effects a higher score, and a sum of (10) is too low for acceptance. And ... now I'm just kind of rambling.


Files in the batch with a rating of:
+ +. ?
None
+ +? ?
(2) Experiment
+.+. ?
(1) Black and White
(2) Death Traveller, Disco 80's
(3) Metropolis, Kirby 64

Responses:

While this may be true, my suspicions regard the level of detail and scrutiny between various judges. That's what's fishy. Beyond quality of the files themselves, you have to inspect it this way - how files were evaluated.

I'm blaming myself here for the fishiness of my group's results... I wasn't clear a lot and I had a hard time putting into words why I thought things didn't feel right and ... yeah.

Nothing but experience. Which means that let's hope that simfilers AND judges learn from each batch judgment.

I know I've learned quiite a bit already, haha...

Also, I know people are already resending files, but from now on, we'd like to ask you to add "[resubmit]" as an extension to the title of song folders, for simfiles that are resubmissions from previous batches.

Alright; def. will do this from now on. I'd type which files were resubmits in the e-mail with the attachment, but being able to see the label is less work than having to read an e-mail again each time you forget details about a file...


The other stuff:
I'll also be editing this stuff into my post with my other other stuff.

"Balanced" rating:
(If one judge's score effects a lower sum, I add the standard deviation, and vice versa)
15 + + +
14 + + +., + + +?
13 + + ? , + + -
12 + +.? , + +.+., + +.- , + +.+?= +.+.+.
11 +.+.+?, +.+.?
10 +.+.-
09 + +?+?, +.+?- , +.+?+?, + +?- = +.+?? = +?+?+?
08 +?+?? , + +?? , +?+?-
07 + ? ?
06 +.? ? , +?? ? , +?? - = ? ? ?
05 + ? - , ? ? - , +.? -
04 + - - , +.- -
03 +?- - , ? - - , - - -

It may seem odd that ? ? - (5) has a higher balanced rating than +. ? - (7), but this is because the standard deviation for the latter is much greater.
To minimize this effect, I curved the standard devation.
crvstdev = stdev * (-(1.079 ^ (stdev^2)) + 2)
With this equation, the greatest standard deviation (2.309) becomes halved.

Curved "balanced" rating:
15 + + +
14 + + +., + + +?
13 + + ?
12 + +.+., + +.?, + + -, + +.+?= +.+.+.
11 +.+.+?, + +.-, +.+.?
10 +.+.-
09 + +?+?, +.+?+?, +.+?- , + +?- = +.+?? = +?+?+?
08 + +?? , +.+.? , +?+?-
07 + ? ?
06 +.? ? , + ? - , +?? ? , +?? - = ? ? ?
05 + - - , +.? - , ? ? -
04 +.- -
03 +?- - , ? - - , - - -

This shows + +. ? in the "Accepted" range and +. +. ? in the "borderline" range, which agrees with what I said earlier. So I guess I'm not so sure about + +? ?.
+ +? +? is the exception here. Because one judge is effecting a higher score, it's true rating is more likely less than 11. (Its curved "balanced" rating is 9.97, which is basically 10 ... but I truncated, so, yeah.)

[Korygon]

Re: Queue/Batch Discussion Thread

If we have 40 already and we just posted the email, I'd say nsane's email had to have had at LEAST 70 or 80. After everyone reads about the new email and resubmits, I figure there should be enough for the batch to close

[who_cares973]

Re: Queue/Batch Discussion Thread

1/4 reject - due to wc being subjective, and quite frankly being a dick, thats all.

not everyone is going to like your file deal with it

[TC_Halogen]

Re: Queue/Batch Discussion Thread

Seems like you only rejected for song choice though. In this kind of judging system, giving a 1/5 just because -you- don't like the song pretty much kills any chance of the song going in.

If they want a different opinion, I can judge it; I don't know the song or chart so it'd be an entirely unbiased review.

If you feel like a hatred/pure dislike is going to get into the way of your rating and cause you to massively change your rating, it might be more worthwhile for everyone to have the song passed on. Basically, do your review and then mark the song saying that it could use a readjustment.

[AsphyxZero]

Re: Queue/Batch Discussion Thread

Where's GD when you need some subjectivemanias.

[Gundam-Dude]

Re: Queue/Batch Discussion Thread

hello

P.S. Next batch, I'll keep my notes short and brief since I'm like the only person other than wc with a brick wall of text for reviews, lmao.

[psychoangel691]

Re: Queue/Batch Discussion Thread

hello

P.S. Next batch, I'll keep my notes short and brief since I'm like the only person other than wc with a brick wall of text for reviews, lmao.

But brick walls are good ^_^ actually I do feel bad because I rushed through this to get it done and didn't really go into too much detail at all. At least on files that need work I think the long notes from judges are good because it helps the person learn especially if they are newer to it.