Expanding FFR's file library: Judging?

**blindreper1179** · 06-3-2008, 09:15 AM

Re: Expanding FFR's file library: Judging?

it takes a while to judge the batches doing it one style. also, shash is no longer with them, it is rebirth now. i see what your saying, but i don't think it'll happen anytime soon. i agree with the "fasions" statement. the files being accepted now will be predicted to be thought awful in two years.

**Tasselfoot** · 06-3-2008, 09:18 AM

Re: Expanding FFR's file library: Judging?

Hey Arch... no offense dude... but you clearly have no idea what you're talking about. For one, Shash hasn't judged files in about 9 months. Initially it was just Shash and myself. Then we added jx. Then Shash left. So we added Rebirth. Rebirth has since quit, but we closed submissions back in March, and haven't needed to replace him until we re-open submissions later this summer.

And, I think EVERYONE who has been part of this process for a long time will tell you (except, apparently, hi19)... we need less files, not more. We had to close the submissions in March because we had 130 accepted files at the time, and we were getting in 100-150 new submissions every 3 weeks.

It will still be an issue when we re-open, and we haven't completely figured out how to fix it... but I've been stressing less leniency for months to jx and rebirth... and I'm probably the harshest of the judges because of it. I also have the final say on every file.

Either way... it will never be a perfect process... but, I can't complain too much. I created this beast. It's my baby. And I'm proud of how many people are so active in creating files for FFR.

**Arch0wl** · 06-3-2008, 09:23 AM

Re: Expanding FFR's file library: Judging?

Originally posted by blindreper1179

also, shash is no longer with them, it is rebirth now.

Then my statement applies even more; I talk to Rebirth a decent amount and have an idea of how he evaluates most files. We generally like the same files but he's much harsher about it than I am, I think.

After reading Tass's post I don't know why what I'm saying doesn't apply unless there's some behind-the-scenes issue that I'm ignorant of, which I very well could be. I know that when you're personally involved with FFR like Tass is it's very easy to see it from a hyper-critical perspective--I definitely was that way with R2, for example--but I don't know why more songs isn't a bad thing, even if the difference is 150 songs versus 250 songs. Granted, I haven't played the files Tass has, so there's that aspect as well, but I'm mostly going by the commentary the judges left on files.

**Tasselfoot** · 06-3-2008, 09:40 AM

Re: Expanding FFR's file library: Judging?

The main reason that I want to keep the queue down, compared to 6 months ago, is because it isn't fair to the simfile guys. There are still a few files from 2007 that are in queue. The process from start to end just takes too damn long.

So the options are to either add more songs all the time OR accept fewer files / allow people to submit fewer in the first place

18 months ago, I was fine with adding more songs all the time. But, we've gotten to a point where there are too many songs in the game. The framework can't adequately handle all of them. So we've had to try to do patchwork fixes... like the page numbers in the genres and the dropdown menu.

2nd style wasn't meant to have this many songs in it. The backend is also not-so-pretty.

In the end, jimerax and I (and rebirth, at the time) have spent a good deal of time talking over what to do about future submissions, how best to handle it. It isn't a simple decision.

I won't even go into the aspect of judging, which was the initial purpose for bringing this thread up... because I can tell you that judging 100-150 files every 2-3 weeks is NOT fun. And extremely difficult to do consistently and accurately over and over again. It gets boring. It gets repetitive. And I don't even test all the files anymore! It was worse for jx and rebirth. But, it is difficult to be a 3rd party trying to critique batch notes without having played the files. It's like trying to review a painting off of a newspaper article.

**Arch0wl** · 06-3-2008, 10:09 AM

Re: Expanding FFR's file library: Judging?

Ah, your reasons make sense. I think the best way free up time for those people would be to get a few volunteer judges (preferably people that are in high school with lots of free time) and give some reward incentive, like credits or a title like "FFR Judge."

As far as the game menu goes, I think the pagenumber listings that you have now are quite effective actually.

**Tasselfoot** · 06-3-2008, 10:49 AM

Re: Expanding FFR's file library: Judging?

Thank you.

And, we don't want a lot of judges. My feeling is that the more people you have judging, the less accurate and consistent it will be. Also, it winds up taking longer, because you're waiting on more people.

Not to mention, you start having people who aren't as good at judging... and you start having more people with access to FFR charts.

I'm still not 100% sure who will replace Rebirth. Will need to finalize that with jimerax. Have a few names though.

**Xx{Midday}xX** · 06-3-2008, 11:04 AM

Re: Expanding FFR's file library: Judging?

Wow.

Just, wow. =/ My already high reverence for staff members just quintupled.

**jimerax** · 06-4-2008, 12:15 PM

Re: Expanding FFR's file library: Judging?

Current FFR file judgement system takes a long time to judge, but it's good for both FFR R1 game and simfile authors.
Especially when judge more strictly/in detail, it will raise the file quality of the game and submitters can get better feedback,
maybe some of them can learn how to make the files better.

But now there're enough files in the game/queue, and people get a lot better at making files compared to 1 year ago,
we want to focus on only the quality of files.
There's no point in judging leniently like arch said anymore, and we'll keep doing judgement that actually makes sense at least..
from both technical and enjoyable point of view.
The judgement style of future batches will be changed a bit anyway.

**hi19hi19** · 06-4-2008, 02:12 PM

Re: Expanding FFR's file library: Judging?

Oh hey Arch welcome to the world of FFR file judging.

Basically, if you want to see a change in the procedure, this is the recommended path to take:
1. find a large heavy book, like a dictionary or atlas
2. place it on a solid surface
3. smash your forehead into it at least 20 times, or until blood is drawn

Now that you have a good grasp of what "ineffectivity" is, place your comment to Tasselfoot. He will do nothing. The end.

**Tasselfoot** · 06-4-2008, 03:01 PM

Re: Expanding FFR's file library: Judging?

yup. go for it.

**Nightfirecat** · 06-4-2008, 03:55 PM

Re: Expanding FFR's file library: Judging?

Yeah... Because of all these files, a lot of them aren't very fun, and they result in us NOT playing them... So more files is not a good solution...

**Villiex** · 06-4-2008, 04:56 PM

Re: Expanding FFR's file library: Judging?

Fun is subjective, so I don't see how you could rate something on it being "fun" anyway.

**Arch0wl** · 06-5-2008, 12:19 AM

Re: Expanding FFR's file library: Judging?

Agree with Villiex.

The files people find fun now were the files that got criticism because they were "too boring" or "not fun" back in 2005--and yes, they made files like they do now back then with some slight changes due to play style, they just didn't use terms like "pitch relevancy" or "layering" to classify those techniques. Preferences change over time and under different environments which is how fashions occur. What people see as a development in "quality" is mostly just a change in fashion; this is obvious if you play a wide variety of music games and see how chart preference varies from community to community. The only consistent factor that exists in any of the commonly played charts for any music game is whether or not the chart vaguely follows the music, which encompasses all of the charts on FFR and all of the popular charts on SM from every era and every style.

Tass's reasoning also seems solid, though. If they're accepting 100+ charts every month out of 200 submissions then it's less likely that they're being strict in the RFPack sense and more likely that they're being practical.

**devonin** · 06-5-2008, 12:30 AM

Re: Expanding FFR's file library: Judging?

I've very very rarely found any files that I just outright don't enjoy playing, unless it's because I just don't like hearing the song that goes with it.

What gets me are often single passages, sometimes even single notes that just aren't placed properly. I'm very very big on pitch relevance and only having notes that correspond to actual sounds in the song (Thus my general dissatisfaction with a lot of mid-level songs with arrow rhythms that aren't met by sounds the whole way (because they'd be too easy)) and it's often been the case that new songs I'm sightreading will just jar in one little spot and completely mess me up.

To me if a song is done well and properly, nothing in it should ever completely take me by surprise. Changing an up to a down because you don't want to repeat the same arrow pattern, or because you -do- want to repeat the same arrow pattern even though the song doesn't go to it is what annoys me the most in a file.

So I mean...even user to user we have fairly vasty differences in what kinds of things make a "good" file or "bad" file to us, so while having only two (three) judges might mean that their preferences on songs make some "good" files get cut, or some "bad" files get in, imagine if I were a judge and wanted to deny a file because "Arrows 154-156 are wrong" and then add in four, five, ten other people all with similar little hooks about their song preferences.

Expanding FFR's file library: Judging?

Expanding FFR's file library: Judging?

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment