The Simfile Quality Debate

[Arch0wl]

Any medium in which preference is involved tends to have undercurrents of the subjective/objective debate. Most people tend to avoid addressing it directly, but it pervades discussion. In food, in music, in movies -- people like something. They know that someone else likes something they really don't like. They have two options: if their liking is specific to themselves, then they have to tolerate the existence of the thing they really don't like. But if they assume universality and promote their preference to a truth status independent of themselves, they have grounds to dismiss anyone and everyone who likes that thing they really dislike.

I will elaborate on my view here later. But first, I need to establish the Rules of the Debate.

Yes, there are rules. In discussing this issue you will be making arguments. Arguments are bound by formal and informal logical rules. If you want to be Right, you need to religiously avoid the fallacies extremely common to this sort of discussion:

1. You cannot ad hominem; this means you cannot argue by attacking someone's character/intelligence/sense, nor can you attack their motive for making the argument. In other words, you cannot attempt to invalidate someone's argument by discrediting the person making it or the reasons they may be making it. The rightness or wrongness of an argument is completely independent of the person making it. If you discredit the person, you haven't actually proven them wrong. To give an obvious example, if a person who makes a mathematical proof also turns out to be a Viagra spammer, the truth of his math proof is unaffected by the horrendous annoyingness of his hobby. I've noticed that the more subjective the medium, the more arguments tend to be appeals to authority -- food critics being the worst. In any case, if you want to be right, you cannot argue this way.

2. You cannot argue through analogy. You can support your argument through analogy just as you can support your argument with examples, but the terms of your argument must stand by themselves without any equivocation.

3. Your argument cannot be simply rhetorical. Linking me to something you think is really bad and asking me a rhetorical question beginning with "do you really think..." does nothing to prove your point. What if I do really think that? Your course of action is then to ad hominem me, which violates the #1 rule of the game here.

By calling these "rules" I am being somewhat misleading. They are "rules" with respect to being right/wrong, in that you can't be right and violate #1 or #2. You can ad hominem or equivocate all you want, you'll just be wrong.

With that said, I'm going to let everyone else state their case first. The assumption of universality (read: quality) is, in fact, an assumption that needs to be proven before you can even begin to talk about "quality" anything in art.

[stargroup100]

Subjectivity and objectivity need to be separated and are distinct from each other. You can like something that is of poor quality, and you can also dislike something of good quality. Quality does not correlate directly with subjectivity. Preference does not correlate with objectivity. You can't say that music is subjective because everyone likes different songs, because this assumes there is no difference in quality between a really good song and a really terrible one. You can't say that food is objective and the food is either good or bad. People have different preferences, and if the food doesn't fit their tastes, they have trouble eating it.

However, there is a universality in all kinds of mediums. The most important concept is the idea of "strong" features. Excluding some very special exceptions, universal appeal applies to anything that is of good quality but does not have any strong features.

For example, take food for instance. Excluding some outliers and people who are allergic, almost everyone enjoys (or at least does not mind) food such as eggs, rice, potato, etc. because these foods do not have strong flavors. Foods like tomatoes, cinnamon, coconut, these have very strong distinctive flavors by themselves, which is why there is a polarization between people who like it and dislike it. Now at this point some of you might be saying "Wait, tomatoes? Cinnamon? Coconut? These aren't strong flavors, I love these foods!" You're missing the point. Someone who has never had a tomato in his or her entire life will probably not enjoy eating a tomato for the first time, because the flavor is so distinct. Most people probably like tomatoes, but that's because they've been eating them since they were young. Some people are more open to new flavors because they tried a lot of them when they were younger, and so are used to the wide range, but the same theory applies. That doesn't mean that tomatoes are bad in quality, it just means that preference towards it is a bit more varied.

Likewise, the same can be said about music. Pop music is popular because there is nothing weird about the sounds, no particularly strong sounds. In this case, the "strong" features or sounds could be atonal melodies, dissonance, glitchy effects, etc. Not everyone enjoys these elements, so music that do contain them won't be as popular as pop songs. That doesn't mean that music with any of these elements is bad, but not everyone will enjoy it.

Charts are the same deal. Some people like dumps and some people like QED's work. That doesn't mean they're good, it just means that some people like them. Again, charts that are more universal don't have anything weird about them, no strong patterns or recurring themes. For example, Drove Through Ghosts to Get Here from DCP is very polarized in terms of people who like the chart and people who don't. It's a chaotic piece and chart with lots of quads in the climax for intensity and miniholds all over the place. Normally, people will immediately dismiss the chart and song upon hearing this description, but upon closer inspection, people who do like it probably understand how the layering and themes fit with the style and intensity of the song. ==Planet KARMA== is almost universally liked because it doesn't contain strong features that the community is unfamiliar with. I say "unfamiliar" because there are some strong themes, but these are ignored by the community because they have seen a lot of it already, and people who don't understand the game can't tell what's actually going on, but they do know it looks pretty. This is also why a lot of people are opposed to the new "art file" movement from post-DCP. The style of these charts encompass lots of features such as color notes and some complex layering techniques that only people who have been playing the game for a while can pick up on and appreciate. To people who can't figure out what's going on, it just looks like a mess of colored notes just to look pretty.

There are rules for each medium to achieve the type of universality I've described, but going past this is a little bit more complicated and there are some gray lines, but in most cases, the distinction between objectivity and subjectivity is pretty clear, and should be treated as two different concepts.

[icontrolyourworld]

There will never be such thing as a perfect simfile that everybody likes in everyway and should be the roll model for future simfiles to be made. Quality is almost entirely subjective most of the time but in the case of simfiles it is subjective so there really isn't anything to debate.

[kaiten123]

Icy, I'm going to challenge your assertion that there is nothing to debate.
the fact that i'm challenging this assertion gives something to debate (the accuracy of your assertion).
therefore there is something to debate and your assertion is wrong.

any attempt to rebut my claims will merely be debating and prove me more right.

though i'm sure that there are other (more interesting) things to debate as well since subjective =/= undebateable.

while i can't imagine the existance of any ultimate file everyone likes in every way, saying that its not possible seems like nothing more than an argument from ignorance. as far as i am aware there is no fundamental law of the universe preventing everyone from liking a single file.



Stargroup, one problem with what you're saying is that i CAN say music (and simfiles, etc.) are subjective. you talk about people being able to like things of "bad quality" or dislike things of "good quality". but your measure of what is good or bad quality is merely your own subjective opinion (or the opinion of someone else, or a group, etc.) to begin with.

unless you have a purely objective way to measure quality in such things a lot of your arguments become little more than your own subjective opinions. (though there may be some useful observations in there, particularly on what things commonly gain mass apeal)

if anyone has a suggestion for an objective measure of quality for simfiles, i would love to hear it lolol.

[stargroup100]

Originally Posted by kaiten123

Stargroup, one problem with what you're saying is that i CAN say music (and simfiles, etc.) are subjective. you talk about people being able to like things of "bad quality" or dislike things of "good quality". but your measure of what is good or bad quality is merely your own subjective opinion (or the opinion of someone else, or a group, etc.) to begin with.

And this is what I mean by people mixing up subjectivity with objectivity. The problem with defining objective quality is tricky because each medium has its own set of standards. However, in general, quality can be described as a degree of excellence, where the piece of work in question shows considerable traits. There are plenty of songs that I personally don't like but I understand why it would be considered to be good, and I do have "guilty pleasures," songs I like that are actually really bad in terms of quality.

Originally Posted by kaiten123

unless you have a purely objective way to measure quality in such things a lot of your arguments become little more than your own subjective opinions. (though there may be some useful observations in there, particularly on what things commonly gain mass apeal)

if anyone has a suggestion for an objective measure of quality for simfiles, i would love to hear it lolol.

Objectivity is not measured in units, because we're not talking about a quantity. We're talking about comparing apples to oranges, not feet to meters. Every chart has its pros and cons, and a different group of audiences that enjoy the chart. Sometimes, the distinction is very clear, like comparing an apple to a rotten orange, but most times it's not, and shouldn't be treated as such.


Anyone that thinks music is purely subjective, I always give them the same argument. Compare the following two songs:
http://www.newgrounds.com/audio/listen/419392
http://www.newgrounds.com/audio/listen/71906

Not a single person can tell me the second song is NOT better than the first one by a mile. If you argue that music is subjective and both songs have their merits, then I'm sorry, you're either deaf or you lack common sense.

If you really want to dissect, the first contains no merits, the vocals are simply an improvised track of random syllables with no thought and the background accompaniment consists of nothing but a bassline which is of poor instrument choice, poor production, and no construction of identifiable theme or concept. The second song, while not musically brilliant, at least does not contain any particularly strong sounds (at least to our generation). The chord structure is boring but solid, and there was a lot of effort put into the production (which is a whole another category of stuff altogether), and the instrument selection had a wide variety of frequencies and tones for balance. Recurring themes are prevalent and there is a traditional song structure, giving the piece a more complete feeling.

Charts are no different. They follow the same line of thinking, same qualities, same ideas, same concepts.

[Reincarnate]

There's no such thing as a universally-bad simfile unless it completely misses everyone's mark. There are just different types of simfiles for different types of people. Some types will be more popular than others and you might find that there are, perhaps 3-5 distinct megaclusters in terms of simfile preferences.

The quality of the simfiles therefore measure how well they live up to the expectations of those clusters. Talking about "objective" judge metrics doesn't make sense when we're talking about preferences. It'll make sense if we're talking about, say, how synced the steps are (a mathematical argument), but that won't necessarily have sway when it comes to perception of quality (at least past a certain point).

I think simfile quality is different from music quality because we judge them differently. Some songs can be horrible to listen to but are hella fun to dance/grind to, whatever. We can recognize that certain charts may be "better-stepped" for whatever reason, but are they as fun?

Simfiles boil down into different perceived metrics of "quality," so I don't think it necessarily makes sense to talk about objective quality unless we're discussing something similar to g-factor in IQ (something that explains the most variance in a factor analysis, in which case we could show statistically which elements constitute the highest degree of "objective quality" where "objective" is really just another form of "strongly-weighted subjective assessments").

[stargroup100]

Originally Posted by Reincarnate

The quality of the simfiles therefore measure how well they live up to the expectations of those clusters. Talking about "objective" judge metrics doesn't make sense when we're talking about preferences. It'll make sense if we're talking about, say, how synced the steps are (a mathematical argument), but that won't necessarily have sway when it comes to perception of quality (at least past a certain point).

Agree with this much, but my point was that preferences are subjective, and should be separated from the concept of quality, which is more objective. Even though the community might have its own perception of what quality should be due to the history of the game (which is still perception), I personally believe there are standards for notecharts just as every other medium.

Originally Posted by Reincarnate

I think simfile quality is different from music quality because we judge them differently. Some songs can be horrible to listen to but are hella fun to dance/grind to, whatever. We can recognize that certain charts may be "better-stepped" for whatever reason, but are they as fun?

We can also recognize a lot of songs that are "better-written" but not as fun to listen to.

http://www.youtube.com/watch?v=n5Yx-ztwrpw

Of course, the educated listener might enjoy this, but a majority of people would find this kind of music boring. Similarly, the educated rhythm game player would find "art files" fun, while a majority of players won't.

Unless you mean "better-stepped" a different sense, such as a chart that manages to layer every instrument but not necessarily be playable. This is not a standard for quality, so it would be an irrelevant (and incorrectly described) case.

[Yesssss]

saying that quality is subjective therefore one should not try to apply objective metrics is a pretty stupid proposition IMO

I'll make an analogy to difficulty. Difficulty is fairly subjective. Some people are better at some files than others. I'm pretty sure everyone will agree that therefore we should not apply difficulty metrics is stupid.
So what objective metrics do we use for difficulty? NPS? It works okay but we should be able to agree that it's not perfect (jacks vs rolls, etc). Even if we are to refine an objective method it still isn't going to fit with everyone, especially when difficulty is somewhat subjective.
Even if you take the same song and speed it up, there are edge cases where it actually makes the song easier to play. I remember a time where I actually had a hard time FCing slow songs because it was so hard to concentrate when the NPS is low enough. Another issue is the area between wristjacks and vibrajacks. I remember when hitting jacks at 140bpm 16ths was harder than 160bpm 16ths. Back when I played one handed, I remember there being parts of songs with holds that were easier to hold when the song was sped up.

Likewise for the quality of a file can be judged by judges (who knew?) by simply having them give a rating to the file. There is your objective metric on the quality of a file. It certainly isn't completely accurate, but it definitely is not completely inaccurate, which is what makes objective metrics on the quality of a file useful

[Arch0wl]

I'll reply to more of this later (girlfriend has made this Harry Potter Day ) but Yesssss's reply touches on something key to this debate:

Originally Posted by Yesssss

saying that quality is subjective therefore one should not try to apply objective metrics is a pretty stupid proposition IMO

It's not that subjectivity makes metrics meaningless, it's that preference for one metric over another is going to be completely down to someone's whim. There's no way to make nonarbitrary metric selection with respect to something like quality, because even your choice of metric is going to be preferential.

In Psychology, you can usually measure the social usefulness of an otherwise arbitrary metric, which is how psychometricists have been able to defend the concept of g (read: IQ) from attacks. In other words, even if it's totally not what everyone thinks is intelligence, it still measures something useful.

But with stepfiles there's nothing you can correlate it with. What could you possibly correlate it with? Popular opinion? If that were the case, you'd open yourself up to a Pandora's Box of scenarios that would take a huge dump on the very idea of quality you're trying to protect. And if you tried to get out of it by correlating it with the existing simfile "elite", you'd end up with a circular correlation: "high quality simfiles are high quality because they correlate with people who think they are high quality."

So when I say "simfiles are subjective" I'm not saying you can't find a way to measure it, I'm saying that any attempt to measure objectivity will just be a means of systemizing your preferences -- you can't transcend arbitrariness because you're still using preferences as a base.

[kommisar]

Different styles of files will always cater to different crowds. This is why there are separate communities developing their own style of simfiles. Though there has always been one globally accepted pattern: structure.

Any file that has a decent structure and has consistency is usually more appealing to the player. Regardless of if musical relevancy or anything of the sort is followed, as long as the file has a specific pattern and follows a set structure for the song, its fun value is already increased tenfold.

[Arch0wl]

But what do you mean by structure? In the PIU and O2Jam communities for example, the structure preferred by the playerbase is extremely loose.

[customstuff]

I think that most people tend to cater to the "elitism" files because they are considered the most accurate. Although there is no rules saying this arrow must go here etc., it feels to some as if it would be playted like this is the computer keyboard were actually the instrument (pitch relevancy...).

I know for a fact that almost every song put out today could be made more fun to play. It could flow better, for example. The problem people have with it is that with many of these patterns it opposes what would be pitch relevant or it could miss a note or two. What I think has happened is that we began with the idea of making it "accurately" represent the song and people enjoyed the idea. However, we quickly made this style into a habit. The reason the people like the files that certain way is because those files feed their habit. It may not have the most fun patterns possible and it could be filled with one-handed trills, but it sounds as if the song would be played in that way. Now it would seem especially strange to skip notes somewhere or play a roll from right to left when the notes go from lower to higher. Just my thoughts on it.

[Yesssss]

Originally Posted by Arch0wl

It's not that subjectivity makes metrics meaningless, it's that preference for one metric over another is going to be completely down to someone's whim.

It is exactly that whim where objective quality is found. The most common elements among people's whim are the metrics.

[kaiten123]

Originally Posted by stargroup100

Anyone that thinks music is purely subjective, I always give them the same argument. Compare the following two songs:
http://www.newgrounds.com/audio/listen/419392
http://www.newgrounds.com/audio/listen/71906

Not a single person can tell me the second song is NOT better than the first one by a mile.

even though i did enjoy the second one more (not by much since the first was funny for a bit), that's still my opinion. even if the vast majority of people like it more its still just their opinions. large groups of people agreeing on something do not make it objectively true. jk, it does, the earth was flat for a long time (when that was the general concensus) and then magically became round later

Originally Posted by stargroup100

If you argue that music is subjective and both songs have their merits, then I'm sorry, you're either deaf or you lack common sense.

cool ad hominem bro.

Originally Posted by stargroup100

If you really want to dissect, the first contains no merits, the vocals are simply an improvised track of random syllables with no thought and the background accompaniment consists of nothing but a bassline which is of poor instrument choice, poor production, etc etc.

all of those "merits" are just your opinion of what makes a song "good". please don't wave your opinions around like they're objective facts.

Originally Posted by Reincarnate

Talking about "objective" judge metrics doesn't make sense when we're talking about preferences. It'll make sense if we're talking about, say, how synced the steps are (a mathematical argument), but that won't necessarily have sway when it comes to perception of quality (at least past a certain point).

+1

Originally Posted by Yesssss

It is exactly that whim where objective quality is found. The most common elements among people's whim are the metrics.

care to explain how people whims are objective metrics? people whims fit the definition of subjective so perfectly. i suggest you look up the words "whim" and "subjective" in a dictionary.

edit: whim: http://dictionary.reference.com/brow...src=ref&ch=dic
subjective: http://dictionary.reference.com/brow...src=ref&ch=dic

[TC_Halogen]

Originally Posted by kommisar

Different styles of files will always cater to different crowds. This is why there are separate communities developing their own style of simfiles. Though there has always been one globally accepted pattern: structure.

Any file that has a decent structure and has consistency is usually more appealing to the player. Regardless of if musical relevancy or anything of the sort is followed, as long as the file has a specific pattern and follows a set structure for the song, its fun value is already increased tenfold.

I was going to make a post similar to Kommi's in this thread, but Kommisar hit it right on the nose BIG time.

Here's where I'll step in -- a lot of simfiles offer a unique AND "stable" pattern structure. If you step away from trying to be as accurate as possible in the transcription of the chart, you're able to control a LOT more. PIU takes -huge- advantage of this, which is why I'll use it to make the example.

- a specific chart difficulty - compare the start point to the choir that comes in, which adds a menacing difficulty spike in accordance to the "drama" or -theme- that the song conveys at this point of the song

- a controllable structure - compare the verse and the chorus; here, the chorus has less musical value, but in an effort to keep difficulty constant, the chorus is overstepped with a clean and consistent structure.

However, there are other factors that make a file replayable.

- speed driven sections - this speaks for itself; the patterns aren't relevant to the song, but they are incredibly diverse and give a player something to not only remember, but strive to be good at.

- gimmicks - while most players try to be great at songs that are clear and straightforward, others go for the challenge of memorizing a song -- some even go as far as using songs like this to immobilize certain players in tournaments.

While pad play has boundaries for what is doable and what isn't, files that are objectively structured properly (and yes, there is an objective property in pad structure because the threshold for what is and isn't doable is considerably different than keyboard) have limitless choices for what can and can't be done. Imagine what can be done when lifting a good majority of those limitations when playing keyboard -- and imagine what can come of 6/7/8 keys if people were just drawn into it properly.

[Yesssss]

Originally Posted by kaiten123

care to explain how people whims are objective metrics? people whims fit the definition of subjective so perfectly. i suggest you look up the words "whim" and "subjective" in a dictionary.

edit: whim: http://dictionary.reference.com/brow...src=ref&ch=dic
subjective: http://dictionary.reference.com/brow...src=ref&ch=dic

oh... I thought that similarities within the biological responses of the human population constitutes as objectiveness whereas the differences were the subjective elements.

Apparently they're both subjective? Is there no word for "similarities within the biological responses of the human population?"

[kaiten123]

Originally Posted by Yesssss

Is there no word for "similarities within the biological responses of the human population?"

the closest thing i can think of would be something like "popular opinion" or "consensus" but one is a 2 word phase and neither is really the same thing.

[kommisar]

debit has it. the pump example goes well actually. they use video/lyric relevancy to accentuate parts of the song.

First thing that comes to mind is i'll give you all my love cz/nm, the end stream goes on while she's running, kinda to simulate the video etc.

since there are no keynotes in stepmania, it's nice to have the feeling you're actually playing the songs with different accentuations.

[Reincarnate]

I'm going to first just outright summarize my view: I don't think you can make an "objective metric." An objective metric is something that is a true indicator of something, independent of preferences. I don't think this applies here.

How can we possibly say "this file is objectively better than this file"? No matter what metric you choose to compare across two files, that metric will boil down to a user-preference which is inherently subjective. Even "structure" is pretty subjective.

I don't think we can treat opinion and quality as completely independent concepts. We usually define a high-quality apple as high-quality because it's healthy, tastes better, looks better, and doesn't make us sick. These all tie into things that give us utility.

The question then becomes "Could we recognize a stepfile as being high quality even if we hate it?" Even if I find Gone With The Wind to be utterly boring, can I still admit it's a quality movie?

I think that's what we're really looking for here -- something that we can acknowledge as being a widely-accepted metric of utility regardless of our own particular preferences. But whatever that answer is, it's not going to be objective. It's going to be "subjective but widely agreed upon."

[stargroup100]

Originally Posted by kaiten123

even though i did enjoy the second one more (not by much since the first was funny for a bit), that's still my opinion. even if the vast majority of people like it more its still just their opinions. large groups of people agreeing on something do not make it objectively true. jk, it does, the earth was flat for a long time (when that was the general concensus) and then magically became round later

I provided an extreme example. There is absolutely no way in which that first song is better than the second. I don't care what anyone's opinion is. It's not about large groups of people agreeing on it, it's about objective standards that define what makes certain sounds pleasing to the ears.

The simple fact is that the earth is round no matter what. The fact of the matter is that second song is more pleasing to the ear than the first song. No general consensus or opinion changes that idea.

aka you're wrong