View Single Post
Old 08-19-2016, 05:46 PM   #1
MinaciousGrace
FFR Player
D7 Elite Keysmasher
 
MinaciousGrace's Avatar
 
Join Date: Dec 2007
Location: nima
Posts: 4,278
Default The Etterna Project

updates cuz this is what you get if you google etterna (rip emma)

client: https://github.com/etternagame/etterna/releases
official site: https://etternaonline.com/




~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
The Etterna project is the newly adopted name for what was formerly known as the "custom build", largely developed by xwidget and myself and with considerable contributions from ixsetf.
The first beta release of the installer is now publicly available at https://github.com/xwidghet/stepmania/releases.

This is an installer release, do not install into the same directory as a pre-existing installation. Currently only the windows platform is installer is available, mac and linux users won't be able to compile until some extra work is done with libs. If you haven't been keeping up with the changes, I urge you to thoroughly read through the release notes. The cliff notes are: difficulty calculator, til death's major features moved internally, default theme changed to til death, advancement towards "configureless" installation, additional general optimization; read the release notes, really! They're pretty comprehensive (though we may have missed a small thing or two).

Now that the project is an officially identifiable entity beyond the nebunomer "custom build" and the considerable work of expanding to a full installer is completed- we are now looking for contributors! If you want to help bring stepmania into the modern age and you feel like you have a the right set of skills to do so, shoot me a pm, post in this thread or contact us via the repository (the latter being the least preferable).

I am specifically looking for:
- Someone versed in theming to aid in development of til death so I can focus on things like calculator development.
- Someone to do some basic graphics such as a new loading splash.
- General/minimalistic orbular/bar noteskins to package with the game in addition to DBZ.
- Files for freely distributable songs to add to the default songs pack. http://www.flashflashrevolution.com/...50#post4494850 for more information.

If you have a general programming/development background and you just want to help out and be a part of the development beyond aimlessly submitting pull requests you should also contact us. We'll be setting up a discord group for contributors and development and a development manifesto as well as planned features and updates will be listed out by priority. I can also direct you to places where my 3 or so months of c++ inexperience inevitably result in probably easily optimizable code.

If you don't have a programming background and you still want to help out, spread the word! About the build! About bugs! Give feedback, and get other people to give feedback too. We do the best we can with internal testing but we don't catch everything. Also we don't try that hard, I mean, most of our time is spent doing development, so your feedback is important. I know it's weird for that to be the case, but it is.

Expect that update and release cycles will be significantly shorter from here on out. Future updates will be install-able into the old ones without any issues, so to use an almost painfully appropriate idiom, you won't miss a beat. As with before all themes that worked on 5.0.12 vanilla are compatible with Etterna. They will require some modification if they wish to take advantage of any of the new features, but nothing will outright break.

Happy smashing.

























** saving original post for posterity's sake **
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
Op
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`

Haha I'm not getting married but it's funny you thought that for a second.

So what then is Wife and what does it have to do with stepmania? Is it a new feature that allows you to experience japanese maid slave porn fantasies in real time as you play stepmania??? Well no, it isn't, but I assure you the entire sm5 dev team is currently working on that as we speak. Moving on to things that matter...

Wife is a rewrite of the existing stepmania 5 scoring system as well as the score save/load system designed specifically for kb4k play. The first aspect is a new scoring system. Any veteran kb player has realized for years that the DP scoring system has fairly significant flaws and that both MIGS and whatever else that other one is that nobody uses are both less desirable alternatives for standard play.



There are a few major problems with the DP system and a number of smaller ones but I'll just highlight the major drawbacks here. At some point I might go much more in depth on the subject but for now we'll keep it quick. DP point assignment is based on the successive timings windows on j4, 22.5, 45, 90, 135, 180ms. Any judgments made within that interval have the same point assignment when the end score is calculated. This means that a 46ms great is worth the same number of points as a 90 ms great, or 1 point.

This doesn't make any sense from a logical standpoint, and the problems are only exacerbated as you continue further through the windows. A 91 ms judgment is assigned 0 points, as is a 135 ms, but hit 1ms later and you are now penalized 4 points (-4). Hit a note at 180ms and you are penalized the same amount, but hit it at 181ms (aka a miss) now penalizes you 8 points instead of 4. Essentially the intervals are too large and the change in points too drastic, creating breakpoints at which players can either be significantly more punished or rewarded based on hitting 2ms earlier/later.

The practical reality is that players are rewarded by the scoring system for intentionally half-mashing sections they aren't sure they'll be able to reliably fc. In the off chance the player overwhelmingly gets goods and greats, he comes out much further ahead on average than he would if he missed a couple notes trying to hit it properly. Now because the scoring system does not align with our perceptions of what takes more skill in the game most players have developed their own quick internal algorithm for judging scores.

I could explain this but it's much easier to just show you. Take these two scores:




Despite the dp scoring system indicating that both scores are nearly equivalent the reality is the second score requires a much higher level of play to achieve, and most if not all of you realized that within half a second. The purpose of a scoring system is to indicate to the player how well they performed. If the system in place is unreliable to the degree that players consistently re-evaluate scores based on the distribution of judgments separately from the scoring system, then it clearly fails in that respect. While it's not as useless as avg nps is for determining how hard files are, it's the same concept at play.

So the question becomes, can we come up with a scoring system that more accurately evaluates a player's performance? The answer is yes, it's actually hilariously easy and it's rather sad that it took over a decade to arrive at this point. The solution is just to not group judgments into arbitrary windows and to evaluate them on the base ms deviance values in the first place. Ms deviance is the distance in ms the player hit a note relative to where it was supposed to be hit. It's simple enough to just fit those values through an equation that spits out a point value for each ms value.

So, what does that curve look like? Well let's start with what DP looks like. DP is not the most robust or accurate scoring system however it's generally pretty good.


(The lowest line is the miss weight, not technically assigned to a deviance value)

And now let's add a curve, say, 1-x^2 (scaled) for simplicity's sake.


Lucky us!

We've already asserted that one of the major issues with the current system is that goods aren't punished enough to prevent players from trying to mash things to improve their score. The differential between the miss penalty and the boo penalty is too great, but that's a result of the use of windows to calculate points. Now, intentionally or not the dp system very closely resembles this curve, and if we made the very reasonable (arguably, needed, and for more than a decade) adjustment of moving boo weights to -1 we would actually have something that almost exactly fits the curve.

This actually gives us a lot to work with. Now we don't want to curve out beyond the 180 ms window, because misses don't have attached deviance values. So we want to emulate the curve 1-x2, and want to curve out to 180ms, such that we start at 2 points and end roughly around -8, or the current miss penalty. The latest possible boo will result in the same penalty as a miss. This also assumes we're working with j4, which was not the assumption I was operating on when devising the initial curve. I'll skip past the development process thus far and just show you what I've come up with since this is already becoming way too long a read. It's tentatively called MSS, or, millisecond scoring.

The idea is to as closely emulate DP while negating its shortcomings, so many of these values should be 'familiar'.
CurveBegin = 18 ms; Deviance values below this point are assigned max points (2). Think of this as the new "perf window"
CurveEnd = 150 ms; Deviance values above this point are assigned the max penalty, or miss weight. Since I was designing this to accomodate j5 and anything beyond ~150ms is considered to be a miss.
linFac = 9.5; Linear strength of the curve
expFac = 2; Exponential strength of the curve
MaxPoints = 2; Maximum point value per judgment
MaxPenalty = -7.5; The maximum penalty. This can also be derived by subtracting the linear factor from maximum points.

curve function:
MaxPoints - (linFac * ((x - CurveBegin)/(CurveEnd - CurveBegin)^expFac)

And essentially it looks like this:

And here are the actual values at each integer deviance value:
http://pastebin.com/UAQgifcB

p.s. For those of you who are aware of this system through streams this variant has stricter parameters

p.s.s attaining 100% is much more difficult using MSS relative to DP however for the moment I'll default to the DP system for the "grade" until I figure out something better/different

The goal was roughly to set the midpoint of each interval on the curve more or less to the j5 dp point value/window counterparts, with the exception of goods which are punished much more significantly. Goods aside if you average the same number of judgments in the first half of each interval as the second half you'll see a negligible difference in your resulting score from j5 dp. Misses are punished slightly less severely and the boo penalty is slightly higher than the previous value at the earliest edge of the window, and significantly more severe at the latest ends.

Now that we have a curve for scoring ms deviance values we need to make a distinction between timed judgments and non-timed judgments, where timed judgments are any judgments that have an associated ms value and non-timed judgments, clearly, do not. Non timed judgements are comprised of misses/mine hits/holds and whatever other junk is in the game.

One of the less significant issues with DP scoring that I skipped over earlier is the ridiculous weight placed on freezes. Now perhaps this is a function of the kb4k playing meta or the kb4k charting meta, but it makes very little sense for held freezes to allot a full 6 points to players. Consider that despite having nearly 1800 "taps" the 480 holds of blue planet make up about 45% of the total points for the file. Silly.

Now freeze pattern difficulty in SM is largely a function of how difficult is it to retain a freeze hold while still hitting whatever else is going happening on top of it. In most scenarios, in most charts, there's virtually nothing going on and freezes are used for accenting specific sounds in charts rather than how they affect how charts are played.

It makes very little sense in this system to award points for held freezes rather than for penalizing points for missed freezes, as you end up giving players free points for doing nothing, which reduces the difficulty of the chart to "score" on without reducing the physical demand difficulty of the chart. Think about it like this — imagine if you got 6 points for each mine you didn't hit. Yeah, pretty stupid, I know.

What I would prefer to do is award 0 points for held freezes, and penalize 5 or so points for missing a freeze. They would operate much more like mines and this would correct the nonsensical nature of their involvement in the scoring system, however in the spirit of not changing things too much, these are the current weights for "non timed judgments" I've set:

Held = 2
NG = 0
Mine = -7.5 (same as miss, like DP)

I'm interested in what people think about the proposal of shifting NG to penalties and OKs to 0, personally I think it makes infinitely more sense however if there's enough resistance to change or a very valid rational for not changing it at all I'm not too opposed to these parameters as is.

Anyway, if you've read this far and you're asking "but what about players who want to stay on j4?"
Well we'll get to that shortly, but it's necessary to understand what the other component of the Wife system is first and why it exists. Creating a new scoring system and establishing rough parameters isn't a difficult task, indeed it only took about 5 hours and that was probably 4 longer than it realistically should have. Jerry rigging a secondary scoring system to the existing game to be calculated at the evaluation screen for screenshots is even simpler and that took about an hour as well.

However that isn't exactly the most appealing solution. The non-integration into the game means that scoring system will only exist within screenshots. The score trackers inside the game won't be able to recall it, and furthermore the timing data that was used to calculate the MSS% will be lost unless it is stored somewhere.

Now afaik it's not possible to use lua scripting to insert the timing data into the game's existing stats.xml structure. Granted I didn't try very hard to figure out if there was, because the immediately obvious solution is to save the timing data somewhere else so that it may be retrieved and used to recalculate MSS%'s for scores, if applicable, upon retrieving the said score from the game's highscore list.

Doing this is rather clunky and it achieves only titular status as a scoring system within the game. Not an ideal solution, and I chose not to continue pursuing it after a few hours of work trying. However it realized an opportunity for me. SM5's current save/load system is the closest thing to a functioning abject failure next to US congress, at least for kb4k play. Perhaps it functions adequately in other scenarios.

At any rate, the distinction between timed and non timed judgments as well as the desire to only need to call a single set of information at a given time was enough to warrant a ground up rewrite/reorganization of the existing system specifically with 4k in mind, aka, Wife. Wife operates in tandem with the existing highscore system and will not interfere with it in any way. The most important distinction between it and sm5's is the introduction of the concept of a "chart". Sm5 employs the organizational concepts of song (reality.wav or disregard) and steps (gate openerz heavy, or oni, or perhaps even a 6k chart on heavy, but all for the same song).

Steps are inherently attached to their parent songs; songs are ID'd by their path in your songs folder. The result is obtained scores persist in the high score table even if the chart has fundamentally been altered, whether by adding or removing notes, or changing pattern formations. Conversely, the same songs with the same steps in different directories have their own separate entries in the high score table. Changing a pack name in any way, such as standardizing the midare pack series titles, effectively deletes any score entries you've obtained on the given files. The same applies for files that are contained in wip or pre-release versions of packs, even if the charts are unaltered by time of official release.

Now this isn't particularly relevant if you just play pad once a month for 20 minutes and you don't give a shit about your development, history, or improvement as a player. On the other hand if you do, the consequences of the current organizational structure are at the very least a nuisance and in the long term, can be frustrating or even infuriating. Seeing as the game is something I and many other players leave and come back to over the course of years, a system designed with longevity and reliability in mind is desirable.

A file is nothing more than a set of instructions given to players through the game, with some additional metadata and an audio file playing in the background. A chart in the Wife system is defined as only a unique set of instructions, independent from song", "steps", and any metadata. It is simply the notes you had to play, in the order you had to play them in, and at the points in time you had to play them at. This information is converted into a string and then hashed to produce an ID for a specific chart. Any .sm files with the same chartkey share entries in the Wife structure. For example, imagine copying reality.wav from hsmp1 into nuclear blast 1. Wife will identify that as the same reality.wav in hsmp1 or where-ever else you have it, and all of them will share the same scoretables.

Now imagine that you do the same with disregard, however in the process you delete the disregard mp3, move reality.wav into the disregard folder, and change the music path in the disregard .sm to reality.wav, and then change the folder name to rasbladabingbong. Just because the mp3 is now "reality.wav", and the folder is now "rasbladabingbong", doesn't mean you're not playing disregard. Wife will recognize that and the chart for disregard will again share the same entry as the disregard file in hsmp1.

However replacing disregard.sm with over shiver mountain.sm will produce a different key, in fact, it would produce the key for over shiver mountain, or rather, for the set of instructions from which the relatable chart for over shiver mountain has been constructed. Altering any set of metadata will have no effect on the key generated. Up to a point, altering bpms will also have no effect. Vertex gamma and vertex beta will not share the same key, however the a static bpm file saved with ddream that results in bpm flux and then again somewhere else but instead using the singlebpm option or using a different editor will share the same key. The tolerance level is currently around 1ms, if any notes stray further than that different keys will be produced.

The idea here is: if it plays the same, it is the same. The goal was to produce something robust enough to catch any "real" (read:affecting play on a practical level) changes in a chart but flexible enough to ignore anything else. Given a few days of fairly dedicated testing it seems the hashing system is respectably good in both areas (read: usable). My guess is it's is manipulable to some degree, but almost always at a disadvantage to anyone trying. I.e. it's much easier to make two things that should match, not match, than to make two things that shouldn't match, match.

While this chartkey system is largely only convenient given the current circumstances, it opens up a large number of possibilities for additional features. A built-in bpm flux corrector that ensures the bpm string is only collapsed while the chart key remains the same, and when/if I feel like returning to the MSD project calculation output can be tagged with chart keys and loaded into the game automatically.

So back to your query about MSS and j4 that you forgot about. The second major distinction between Wife and the existing system is that it records your ms deviance record for anything you play along with your judgments and any other standard data output. I know, this isn't exactly revolutionary and honestly it should have been done after we all stopped using pentium IIs, 64mb gfx cards, and floppy disks to store data.

Despite MSS being designed for j5 you can still use it while playing on j4, though it will weigh anything in the 151-180 ms range as a miss (so basically the majority of the boo window). This isn't particularly desirable for players unused to playing on j5 and switching to MSS might be too daunting or even just frustrating. However the entire score calculation system has been rewritten to calculate based on standardized input designed to accommodate ms based scoring. This means that any number of scoring systems are supported whether they emulate MSS or DP in methods of calculation ad infinatum. This means it's trivially easy to support a j4 targeted variant of MSS that looks something like this (eyeballed for now):

CurveBegin = 40 ms
CurveEnd = 180 ms
linFac = 9.5
expFac = 2
MaxPoints = 2
MaxPenalty = -7.5

Behold MSLite, aka "waifu". Now perhaps the image of this comic is flashing at the back of your mind https://xkcd.com/927/

It's not a problem, since the original ms deviance record is being stored and the score calculation is standardized, Wife will calculate and store percent values for each type at the evaluation screen. It will then only display the value for whichever scoring type you have currently selected (along with indicating what that type is). Switching from "waifu" to MSS will simply change which value is displayed. So will switching to and from any combination of DP/MIGS/MSS and so on and so forth.

The judge value under which scores were achieved on are saved as well as the parameters. Wife will check for changes in either; changing from j4 to j5 will recalculate dp/migs/whatevertheotheroneis. Changing the parameters for any of the ms based systems will also invoke a recalc using the new parameters.
Note: haven't actually implemented the conversion detection yet but that's not a priority since everything it needs is already being saved

Obviously, this only applies to scores attained after Wife is operating. Pre-existing scores in the games highscore table are converted into Wife's format, the percent values for each scoretype are calculated with the assumption that they are achieved on j4 (they have to be, since no deviance data exists). Ms based scoring systems will be ignored and nothing will be calculated.

Currently only dance-single type charts are compatible with Wife. It should do nothing if solo/whateverelse steps are selected.

Eventually I'd like to make components of the system seamlessly compatible (as much as that's possible) with other themes however I'm designing/testing/building it in conjunction with what is turning out to be a complete rewrite of my theme and the two are necessarily interconnected.

There's more to say I think, and I'm sure there are questions I forgot to pre-empt, but this about covers all the major points. I'll have a public alpha for download and experimentation linked here in a day or two which will be inside a version of my theme stripped of anything that will alter configurations.

I think it's about time we moved stepmania past 2003

ps. if you're wondering why i named it wife, say MSS 3 times quickly

Last edited by MinaciousGrace; 09-13-2017 at 12:09 AM.. Reason: public installer release
MinaciousGrace is offline   Reply With Quote