Queue/Batch Discussion Thread

[kommisar]

when I judge, most of my criteria for accepting is fun value, the rest is structure. I don't really look at the song itself since that's VERY subjective, but if the song itself generates very long repetitive sections then it's a given


who didin't give staring at my spaceship a +?

oh aj. u haterrrrrrrrrrrrrr




here's what I usually base a 1-5 scale on (25 point average)

5 points for creativity/originality
10 points for fun/replay value
5 points for structure/technicality
5 points for flow/patterns

although it's obvious that some are more important than others, this usually works out. the reason we have multiple judges is to avoid any kind of huge bias. if all the judges reject your file, there's a pretty big chance most of others will agree too. I can think a certain pattern should be done one way, next judge says otherwise. there's no way to eliminate different opinions aside from getting input from several people.

[who_cares973]

i try to be as much of a robot as i can be when it comes to judging

[Xandertrax]

Ouch, don't appreciate the jab, Patashu.

and wow. 6 pages of ffdramalution. I missed this stuff. Not really.

The main problem is that 4 judges is far too small of a sample size to tell how 'good' a file is. Howabout, in hotly contested files that get ratings across the board, you bring in another judge or two to get a few more ideas? Not sure that will accomplish much.

Also, was there a final conclusion on Feux d'Artifice? It was thrown in there but I never saw the answer come up.

[jimerax]

Accepted files of this batch aren't finalized yet, just files that got 17 or more points are safe for sure. A few other files could be accepted, and we need to determine acceptance criteria for them. going by ratings, or using secondary judgement system.

I'm not thinking of conditional queue for borderline files (this is for files that are accepted but offsync/bad parts or features).

And yeah being subjective isn't always bad but putting too much weight on how you hate/like the way of stepping, the song, difficulty causes judgement inconsistency, which isn't a good thing as judgement.
I don't think all judgement should be completely objective since basically everyone has different opinions, but we need some standards for judgement for reducing inconsistencies.

[psychoangel691]

Originally Posted by kommisar

here's what I usually base a 1-5 scale on (25 point average)

5 points for creativity/originality
10 points for fun/replay value
5 points for structure/technicality
5 points for flow/patterns

See now this is actually a really good idea I don't know why I didn't think about mentioning something like this sooner. This actually reminds me of how judging was for the skate competitions that we had at the skatepark. It was like a judge would rate on the difficulty of a trick, originality, how technical it was and like the wow factor then it was averaged.

Why don't we do something like this judges look at those different aspects on a 1-5 rating, rate each of those and then the final will be the average of all the aspects?

So based on Kommis aspects he mentioned say a file was really fun it gets a 5 for that, has maybe a few technical errors so lets say a 3, overall flow is pretty nice but maybe a few off patterns giving it a 4 and we'll just say 4 for originality here. Which ends up giving it an overall rating of 4. I think that's a really nice idea. It still gives a ton of room for opinionated things but it's balanced out.

[ichliebekase]

Originally Posted by psychoangel691

I really think there needs to be some sort of guidelines put in place to avoid extreme differences in ratings such as the + and - on the same file because that's just a huge difference.

This :/ on two of my files, I got ratings of 1,2,3,4, exactly that. Only one had an even rating of 2,2,3,3.

I also wanted to say before that what i love you said about the file he judged on mine, he was out of line for saying things like "LMFAO" and I think "Seriously?" was thrown in there too, and he completely rejected the file. It was pretty rude the way he said it all. Also, I think it was the same file that Carlos had judged, and he flat out accepted it with a +.

Something was seriously wrong there.

[krunkykai22]

Originally Posted by Izzy

yea except they have to be perfect to "them". It's pretty much entirely subjective after the point of the file being synced and not having notes go to nothing etc..

I like files being different then them so oh well I guess.

Originally Posted by psychoangel691

I was going to say pretty much the same thing. What actually defines "perfect" for a file? With such a vast difference in opinion on subjective things you can't even really call something a "perfect" file.

Originally Posted by TC_Halogen

I was subjective, but I'm capable of admitting that I was, and if you look at my notes, you'll even see that files with SUBSTANTIAL AMOUNTS of notes/fixes requested still ended up getting no lower than a [+?] because I'm able to separate between what should be there in my opinion, and what should be there in accordance to how the rest of the simfile is. Stop acting like I'm reacting like this because of this file - there's other files from the previous batch that got knocked down a ridiculous amount for stupid reasons, like song choice. There's even some files that had small amounts of errors and still got a [-].

But hey, I don't know a thing about judging or whatnot. I'm arguing with someone who thinks their opinion is right and nothing else because they couldn't even bother to address the issues that I pointed out -respectfully- like I asked to.

I'm done judging FFR files - if people are going to sit here and cut each other's throats off because they can't tell the difference between judging off of their preferences and what's right, then what the hell is the point. I love how the argument of subjectivity is coming out entirely because I choose -now- to speak up, rather than trying to focus on why people's votes are too -overly- subjective.

The rest of the judges can go have fun doing what they want. I feel sorry for the really strong judges who are consistently good because they know what they're doing having to deal with ratings that deviate ridiculously without justified reasons whatsoever. Apparently, getting a legitimate discussion when I actually asked questions about how to FIX something is a complete waste of time because some judges won't even bother to give a straight answer. What's the point?

Seeking perfection in simfiles is going to get you ABSOLUTELY nowhere because the users who are playing your game will not notice a good amount of the details that people are putting in.

Find another judge for me, I'm not judging on this team until I see a change in people's attitudes. It's not everyone, but there's people on the team who I simply don't agree with being on it for reasons of not being active with the site, not caring about the game, or not legitimately wanting to help someone out because they feel like flaunting around their elitism when no one gives a shit.

1.) You guys lost a judge.... A damn good judge at that. Just because he wanted his gf to get her first FFR file in game doesn't mean he is just defending her for the hell of it. Aj has a lot of good stepcharts for both FFR and SM AND ITG.... I think he knows what the hell he's doing. For him to actually go over and shadow things on that file, I'm sure he damn well listened more in depth than anything.

2.) Personally, I LOVED The Final Hour. I'm not judge, don't get me wrong, and I don't have a file in FFR, rather 2 1/2 on the DF Engine, but I think what Kayla did was ****ing perfect. That's my opinion, she did what she did because SHE felt it to be the best file she wanted it to be. Not to mention, when that file came out in one of the DF Tournaments when FFR was down, did you guys see the damn compliments? EVERYONE was telling her how amazing the file was. I don't give two ****s if they aren't FFR Judges or anything, take that of the community in with your own reasons. If you just reject the file based on one/two/three peoples mentality what's that to say of the community?

Here an example:

AJ creates a Stepfile for a rock song "Disturbed = Believe." 99% of the community LOVE the file that he had shown to them offline. 3/4 judges come in and give it an 11. Don't you think the community would be like "really? WTF Elitist assholes."

If the file was completely retarded don't get me wrong. Overstepped, off sync, notes being filled in empty spots, BPM changes ridiculously ....whatever, than I understand it. But, if it IS a good file to us (the community) Than why not just suck it up and put it in? What makes The Final Hour different from .... For FFR? Or any other Synthlight songs? Yeah they are history for FFR, so why can't The Final Hour be?

3.) I agree the strictness needs to subside just a bit.

4.) Kommi's rating system is beast. Personally that's a HUGE thing I believe and apparently with others, would be fantastic in judging. But, still, there will always be subjectiveness.


I personally don't know what more to say about it. Somehow I knew that because Kayla is Aj's fiance, that would get thrown into the mix. I'm completely against that. 100%. Kayla knows what she is doing, and so does AJ, no need to get personal about it. nuff said.

[psychoangel691]

Originally Posted by ichliebekase

I also wanted to say before that what i love you said about the file he judged on mine, he was out of line for saying things like "LMFAO" and I think "Seriously?" was thrown in there too, and he completely rejected the file. It was pretty rude the way he said it all. Also, I think it was the same file that Carlos had judged, and he flat out accepted it with a +.

Something was seriously wrong there.

That's another thing that really shouldn't be happening. I don't think at any point a judge should say things like this sucks or laugh at people. It needs to be kept in mind that some people are newer and they don't deserve to be treated like that when they are trying to learn.

Actually something I'm noticing and I see you lurking Byron and don't take this the wrong way because I'm not stabbing but I noticed you mention things being offsync but I didn't see anyone else mention it especially Carlos and he's usually the one to nail that stuff with ddream. On DF he was always pointing out that people's gaps were wrong. It's just really weird to me that sync seems to be coming up as an issue for you but not others. Maybe there's something up with your computer? I know when AJ would do stuff on the one desktop we had the sync would be all screwed up and yet when it was done on the laptop it was fine.


Just throwing this back up quoted since it ended up back a page and sometimes people don't go back

Originally Posted by kommisar

here's what I usually base a 1-5 scale on (25 point average)

5 points for creativity/originality
10 points for fun/replay value
5 points for structure/technicality
5 points for flow/patterns

Originally Posted by psychoangel691

See now this is actually a really good idea I don't know why I didn't think about mentioning something like this sooner. This actually reminds me of how judging was for the skate competitions that we had at the skatepark. It was like a judge would rate on the difficulty of a trick, originality, how technical it was and like the wow factor then it was averaged.

Why don't we do something like this judges look at those different aspects on a 1-5 rating, rate each of those and then the final will be the average of all the aspects?

So based on Kommis aspects he mentioned say a file was really fun it gets a 5 for that, has maybe a few technical errors so lets say a 3, overall flow is pretty nice but maybe a few off patterns giving it a 4 and we'll just say 4 for originality here. Which ends up giving it an overall rating of 4. I think that's a really nice idea. It still gives a ton of room for opinionated things but it's balanced out.

[woker-X]

Let's see...

Can Can - Yay

Capriccio No. 24 - Only problem is the song quality, someone has a better audio for this song?... Or have anyone seen Scintill (because he did the audio file)?

Four Blocks Madness - Fixing

Sexplosion - Fixing

TtMH - Fixing

awww... borderline files should be conditional

[i love you]

Originally Posted by TC_Halogen

Everything above the next quote are all gibberish BS due to my anger and disagreements.

*yawn*

Okay, now to actually answer something that you or anyone else from DF didn't know about:

Bold point #2: I don't even know why you're coordinating stuff within the game in the first place when you've been gone for who knows how long. That's my honest opinion, since you've been so kind enough to be abrasive when this entire thing could have been avoided had you actually discussed what I asked for and not made a snide comment. In my eyes, being a judge is a privilege, and not a job - if your attitude is going to be complete garbage to others, then you shouldn't serve in the community to begin with.

I haven't been gone from the game, i just don't play competitive FFR anymore which is why you haven't seen any scores from me in years. When i do tier points, I play-test each file offline plus look at the top 200 scores seeing what other people got in the file. i guess when you hear the word "retired" you will assume that i don't play the game at all which is true but that isn't going to stop me from playtesting files. I think you once told me that you were able to view replays from other players on your site so how about you go and look at my replays and tell me the last time I've played. I'm pretty sure it wasn't recent but it sure wasn't forever ago. Also, my attitude towards the community, and my judging are mostly positive.


Anyway, back on topic:

Originally Posted by kommisar

when I judge, most of my criteria for accepting is fun value, the rest is structure. I don't really look at the song itself since that's VERY subjective, but if the song itself generates very long repetitive sections then it's a given.

This is very similar what i do which is the main reason why i play-test each file first to see how fun and replayable the file is. If i notice something while playing it, i would mention it within my notes. Song choice, is usually minor for me but if the file is repetitive then i would say something about it and that IMO is what makes the person lose a lot of points from me. In the editor, i mainly look at sync, inconsistencies, and pattern choices. Of course, there would be times when I'd be subjective here and there but that doesn't mean that the person should agree with everything. If i think something could be better, i would state it but that doesn't mean it would affect my overall judge about it.

[customstuff]

I might have a very small amount of fixing. I COMPLETELY disagree with jacks like 444333222 being awkward. Yes, 44334433 is awkward and I could try other patterns, but the first one is NOT at all.
I guess I just need to check up on my 12ths etc., make the congas single jacks, and redo that one pattern and I should be good to go :P

I really want to know how many points +?, +?, and + would be though.

[Coolboyrulez0]

It took me multiple years to get some files into the game and I never whined and complained like this,
it encouraged me to work harder to produce not only fun but accurate files.
Jesus Christ, fix the file, or step a new one, big ****ing deal.
Everyone needs to stop whining and needs to start moving on.
Really, I don't know why I had to read 8 pages of bitching this morning.

Happy ****ing Holidays

On that topic, expect some videogame nerdcore files next batch

[krunkykai22]

BUT, why fix a file YOU feel along with others is awkward?? That's retarded, when you step a file you step it to comfort along with what's supposed to be there, making a chart awkward that you DONT like is basically not your file then if you just do what others want you to do, come on now...

[Coolboyrulez0]

Originally Posted by krunkykai22

BUT, why fix a file YOU feel along with others is awkward?? That's retarded, when you step a file you step it to comfort along with what's supposed to be there, making a chart awkward that you DONT like is basically not your file then if you just do what others want you to do, come on now...

Please form a cohesive and understandable English sentence, then get back to me.
You are 24, according to your information, and I can't understand a single thought in this "rant".

[Plan_Bsk81127]

Originally Posted by Coolboyrulez0

Please form a cohesive and understandable English sentence, then get back to me.
You are 24, according to your information, and I can't understand a single thought in this "rant".

You are asking for a post with many stars brah. :P

[psychoangel691]

Originally Posted by Coolboyrulez0

It took me multiple years to get some files into the game and I never whined and complained like this,
it encouraged me to work harder to produce not only fun but accurate files.
Jesus Christ, fix the file, or step a new one, big ****ing deal.
Everyone needs to stop whining and needs to start moving on.
Really, I don't know why I had to read 8 pages of bitching this morning.

Happy ****ing Holidays

On that topic, expect some videogame nerdcore files next batch

Again things were misunderstood. It was never about a particular file it was about feeling that the standards are too strict to get a file in game. Things got blown way out of proportion on something that shoulda just been a discussion on how people felt about the current system and how it was running. The current system feels extremely flawed to some of us but not to others. No reason it can't be discussed. I really like the idea of rating different aspects of the file then averaging it for a final grade. I really think that can help alleviate some of the issues.

[Coolboyrulez0]

Originally Posted by psychoangel691

It was never about a particular file it was about feeling that the standards are too strict to get a file in game.

They are just fine, it appears you struggle to accept criticism and subjectiveness.

Originally Posted by psychoangel691

Things got blown way out of proportion on something that shoulda just been a discussion on how people felt about the current system and how it was running.

Totally agree with you here.

Originally Posted by psychoangel691

The current system feels extremely flawed to some of us but not to others.

It appears we are on opposite sides of the spectrum on this and seeing as this is your opinion I guess I'll have to accept it.

Originally Posted by psychoangel691

No reason it can't be discussed. I really like the idea of rating different aspects of the file then averaging it for a final grade. I really think that can help alleviate some of the issues.

Interesting concept, I do believe this would make judges spend more time than they currently spending which is down to the discussion of the balance quanity vs. quality.
Being an artist that has submitted files since '07 I would say the current system has a very good balance of quanity vs quality, and sure, there always is room for improvement... but I think we are currently fine with the system that is in-place.

[krunkykai22]

Plan xD I won't make a post with stars no worries xD

[psychoangel691]

Originally Posted by Coolboyrulez0

They are just fine, it appears you struggle to accept criticism and subjectiveness.

I don't struggle to accept it and I'm not even close to being the only one that feels this way about the strictness. I feel it's a major issue when files are getting rated with a + by one judge but a - by another. That's an issue there's no reason a rating should be that different from another. Which is why I say that we need something to balance the system out.

Interesting concept, I do believe this would make judges spend more time than they currently spending which is down to the discussion of the balance quanity vs. quality.
Being an artist that has submitted files since '07 I would say the current system has a very good balance of quanity vs quality, and sure, there always is room for improvement... but I think we are currently fine with the system that is in-place.

I really don't think it would cost too much time. With the new system judges aren't doing a batch of 100 files like they used to. Being that it's now on average around 25 files a judge has to look at that's not bad at all. Plus it takes more time to nail down technical errors (which is what judges are doing now anyway is pointing out every error) then it would be to just give a rating for how fun a file is. So really adding the other aspect ratings shouldn't cost much more time at all.

I just think it would make a good balance since there are such vast differences in opinions on the judging panel right now. You've got people that are just super harsh and others that will let a lot slide.
Another thing I noticed is it seems that if a file has errors but was created by a stepartist that has been submitting for a long time or if it's a judges file that people will let it pass even with errors like mis-rythms but then if a new stepartist comes along with minor mistakes they are hit hard for them.

But we'll have to see what JX thinks he's the main man here. .

[ichliebekase]

Originally Posted by Coolboyrulez0

Please form a cohesive and understandable English sentence, then get back to me.
You are 24, according to your information, and I can't understand a single thought in this "rant".

What he said was why would you step a file according to what others tell you to do? Would it then be called your own file? Also, if someone told you to step something that would make it seem awkward for you [the stepper], then why should you have to put it there? People step to what makes the file seem worthy of being accepted in their eyes. That doesn't make it wrong.


Also, this is the attitude that everyone is talking about. Why does everyone have to have a stuck-up, snobbish attitude towards everything? It doesn't make any of this fun anymore. At least when I was stepping for DF I didn't get attacked from Kayla, AJ and Carlos every single time.