Definition of a FFR AA
 

This is a continuation thread based on the feedback from the FC implementation thread. Please go read that thread to know why this metric seems to be needed on FFR.

So, what is a AA ?

Basically a AA on other games serves the purpose of determining if you did "well enough" on a given chart, without necessarily being close to perfection on it. On Stepmania for example, this grade is obtained by getting 93.00% accuracy or better.

What is accuracy on FFR ?

Since FFR is frame based, it makes sense for its judgement values (Amazing, Perfect, Good, ...) to be defined by some frame window around the exact note timing. In any case, frames can always be converted to ms timing. That being said, the simplest way one could define accuracy on FFR is by comparing the obtained raw score on a chart with the AAA raw score on that same chart. This allows for any configuration of non-perfects to be a AA, as long the raw score is sufficiently high.

Other games like Etterna will consider even the slightest ms timing difference to compute the accuracy. In other words, a 100% is extremely rare and only achievable on super easy/short charts by very talented accuracy players. In the context of FFR, that would mean considering Amazings in the computation, where an Amazing would result in more points than a Perfect. Given the reputation of FFR to be relatively unstable at that scale, I think it's safe to say that Amazings should not make a difference.

When it comes to a specific % requirement, I believe it can be discussed mostly subjectively. I personally don't see why we couldn't use a round number like 90%. One interesting piece of data that could be useful is how much % can one get on average when just mashing charts. This would give a lower bound to the discussion. I don't have the time right now to compute that, but if someone could, that'd be very helpful.

What about chart structure ?

Chances are you havent asked yourself that question yet. Chart structure matters when you compare two charts, and one has a single difficulty spike whereas the other one is more balanced overall. This currently affects FFR in a major way since the AAA metrics that we use do not account for such chart structure, resulting in what is commonly known as "farm files". Well, the same logic applies to any AA metric, except that it introduces even more of these "farm files".

Although the AA metric would be even slightly more "broken" than our AAA metrics, it is far less of a big deal. The reason is that AA would never be FFR's main skill measure. Knowing that, there's no real need to fix AAA stuff before implementing AA. There's a non-0 chance that if/when difficulty is redefined in the future, AA will have to follow the changes, but that's probably the least of our concerns given the other implications that would come with a redefinition of difficulty.

What would AA give to users ?

Obviously, the first thing associated with AA's would be a completion metric just like the AAA's and FC's (and maybe SDG's eventually). Then, some Tokens and Skill Tokens could be made to use this new tracked grade.

________________________________

So what do you think of this suggestion, and do you have anything you'd like to see regarding this AA grade ?

Re: Definition of a FFR AA
 

I think the most reasonable way to set an AA threshold in FFR is by using raw goods- similar to SDG. if AAA is 0 raw goods, and SDG is <10 raw goods, maybe somewhere between 30 and 50 raw goods? anything stricter than 30 is too close to SDG to really be valuable. 50 is a strong choice because at that threshold you're at least demonstrating some amount of control and competency on the patterns- maybe there was a bit of mashing on a difficulty spike and some bigger mistakes here and there, but you were doing something right for a good portion of the file.

(but if you think you can implement a competent accuracy% system for this, that'd be even better. i just think raw goods based fits well with the existing grades)

Re: Definition of a FFR AA
 

Seems unnecessary tbh lol. Isn’t SDG basically a AA? And unless u want to score lower for tokens, that’s literally just letting the better scorers AAA until the end when they can get that I don’t play competitively often for myself, so outside looking in: it’s pointless aside from the clout of saying u got a AA. I’m know SDGs have their own reason for be excluded from this kind of scoreboard reasoning since it’s a kind of a 1/10 get if u can’t. How would u determine a AA? Getting more perfects than amazing? Would it be a percentage? Also if u do implement it, maybe make it like “perfects and amazing -only score”a limited to that same ‘1/10 get’ before the AAA. In the end if it does happen I’m looking forward to seeing my scores change and people get hyped since others will use it to work on their songs. If in the end the AAA metric is all that matters, I’d just kind of cut the idea unless u wanna hype competition. I also have no idea what metrics are like in these games coz I just go and play em, I mean as far as charting goes for this, I don’t see the relevance since for all I know judging is subjective too. Maybe it could help sort out more songs in the engine since there’s over 2k. Regardless it’s cute seeing everyone want to expand the game and I’m down to see your progress!!

Re: Definition of a FFR AA
 

Why an absolute raw good count vs a relative one though ? To me it seems like absolute count would simply be too similar to the other flags we use (Blackflag, Booflag, SDG), whereas a relative count would bring in some variety to FFR while also catering to the players from other games.

Re: Definition of a FFR AA
 

A value of 95% of the maximum possible raw score would sound good. It's a nice round number, high enough to show you managed to play the file decently, and low enough to be significantly easier than SDG. A value of 90% would be too low: looking at my past scores I've already mashed my way through some difficult files, getting 90%.

Re: Definition of a FFR AA
 

Oh nice, can you please give some examples of those scores in here ? In fact, after some discussion on the % on discord, 90% definitely IS far too low for most charts. When I get access to my laptop I will compute the average raw score % that a SDG represents on all files, and feom there we could adjust the AA % to be something lower.

Re: Definition of a FFR AA
 

The tier points page gives a nice comparison of obtained raw score and maximum possible raw score for a bunch of files. I see I've managed to get a 93885/104000 = 90.3% score on System Doctor. Not sure when I got it, but to compare: I'm mid-high D4 now, and should definitely NOT be able to get AA on a difficulty 101 file.

Other examples:
92.8% Revolutionary Etude
90.3% A Quick Death
92.7% Grist
91.0% Magical 8bit Tour
91.0% Schmollbluk
... and more.

For a 1000 notes file, a SDG equals a score of 99.51+%.

Re: Definition of a FFR AA
 

Great, thanks for the info ! A more accurate estimate has been made at around 97%, considering an SDG on a 1500 notes chart is 99.333%. On that same notecount, a 97% would mean 90 raw goods, and on a chart with 3000 notes it would be 180 raw goods.

Re: Definition of a FFR AA
 

My bad if the raw goods counts in my examples are wrong, typing quickly on phone.

The idea remains that the % for AA should be agreed on w.r.t. the average notecount to be as representative as possible.

Re: Definition of a FFR AA
 

I can't see a better way than this...though you will still have those anomalous files that have a huge skill gap to go from AA -> SDG (System Doctor is a great example, very farmable as it is for skill rating lol) and then other files that you can improve drastically on with a small skill increase (stamina intensive and marathon files)

I don't see anything wrong with the idea though since all we're doing is counting the AA's and not assigning ratings to them based on difficulty ;)

Re: Definition of a FFR AA
 

Yeah the flaws of our AAA system shouldnt be applied to this since it would purely be a rough measure of "did you get a somewhat decent score". It still suffers from the same-ish chart structure I mentionned in the OP yes, but I dont think it's important given the goal of the metric.

Re: Definition of a FFR AA
 

If the primary goal of introducing AA is to replace FC counts, I disagree. In other rhythm game communities, AA is simply just a grade received on your performance of the file. This can cause more confusion for new Etterna players thinking "wtf why is it so easy to get an AA on FFR lmfao?" If you really want to introduce AA, you might as well start introducing other grades like A, B, C, etc. in this definition as well.

However, any hypothetical AA, A, B, C, D, etc. is not contributive to your skill rating. It's only just letters designed to unofficially and unnecessarily validate your own skills while playing this game. I say unofficially because these grades are not factored into the skill ratings computation and I say unnecessarily because your scores should be enough to make you feel proud.

If the skill ratings formula is revamped to somehow incorporate these grades, then my vote would have switched to "Yes". Right now, the AA proposal just seems like another one of millions of random stats we currently have on this website lmfao.

Re: Definition of a FFR AA
 

No it's not. It came from the FC discussion, but it's not related to FC at all.

If we start doing everything exactly like other games do, what's the point of being a different game ? Sure there's common sense like don't make "AA" mean you passed a song, but the point still holds.

Firstly it's not for you to decide how others perceive/gauge their performance; we have a main metric in FFR which is skill rating, outside of that can live other metrics to diversify the game.

Secondly, implying it's necessairly a pride thing is close minded: why can't people just have Fun trying to track how many files they can play decently on with a structured, automatic metric ?

Ok, your point is simply "I think we have enough secondary objectives". That's fine, but it's entirely subjective. If you dislike seeing more of these which arent mixed with the main metric, you're free to say so. Otherwise, there's no point in forbidding others to have this metric they find interesting/fun for whatever reason.

Re: Definition of a FFR AA
 

I dont know what this adds to ffr. I am not very understanding of everything, but ffr is a aaa game, you goal is to get a aaa, not a aa, so adding a aa type of score feels like its not ffr.

Re: Definition of a FFR AA
 

Just like tokens, skill tokens, tier points, badges or any other secondary element of FFR that is not the main skill measure, AA would give another optional metric to track/work towards.

Re: Definition of a FFR AA
 

I guess, its fine, but it feels like you could just calculate it on your own? idk why getting a aa is that important for ffr.

Re: Definition of a FFR AA
 

2% raw goods / total notes in song

Re: Definition of a FFR AA
 

You could also count your AAAs by going through the list in level ranks of your profile.

It's not -important-, it's a suggestion to implement a feature which may incentivize some people to work towards instead of focusing on AAAs. Nobody asserted that it's a necessity nor that it will be relevant for all players (just like the other optional things I mentionned that FFR has).

Re: Definition of a FFR AA
 

Fun fact: FFR did show these AA/A/B/C/D/E rates already on the results screen, but then it got removed. I don't remember how it exactly worked, it likely went off combo score. They weren't tracked anywhere else AFAIK.

Re: Definition of a FFR AA
 

Yep, I'm aware of that :p

They were only part of the game engine and not tracked anywhere, so it seems very normal that it never was interesting enough for anyone to care about.

Reason why I only suggested some kind of AA equivalent is twofold:
1. On FFR, as we've discussed in this thread, you can relatively easily mash your way to a % as high as 95%-ish, because of how our judgements work. Therefore anything worse than that AA metric would converge to the same required effort to obtain for even less significance than the AA compared to AAA.

2. In some other games, AA is a very common metric for players to determine if they did decent on a file. Very rarely do we see players go for the lower grades.

Re: Definition of a FFR AA
 

I can see an A rating for 95% or 96% working along with an AA rating of 98%, but more than that indeed seems redundant.

- A rating: shows you played well.
- AA rating: shows you played great.
- AAA rating: shows you played perfectly.

It's true AA would be a more popular metric but I don't see a bad thing in adding A too.

Re: Definition of a FFR AA
 

I gave the idea some thoughts, and although I personally dont disagree with it entirely, lets focus on AA only for this thread. You know, if AA is ultimately deemed to be a bad thing to add, A will certainly not be better.

Re: Definition of a FFR AA
 

If I'm interpreting this correctly then this is essentially what I'd been pitching as well: 99% max raw score. A 2000 note chart produces a score of 100k for a AAA, so setting the AA bar at 99k allows for leniency of 40 raw goods.

I don't think it's an issue setting a stricter AA% grade relative to other games with similar metrics given how acc-focused FFR has been for so long, in addition to the already wildly high raw score percentages a player could obtain mashing through most of a chart. Setting the threshold looser than 99% very quickly takes a nose dive when the point of the metric is to establish player competency in playing a chart. E.g. establishing AA at 98% max score doubles the raw good threshold to 80g on a 2k note chart. I feel many players would argue a major lack of file competency is still present if someone is just skirting past that proposed AA line with 77 or 78g.

Obviously charts with very low notecounts run into the issue of having a AA grade that already forces a SDG or better. But for the large majority of charts in game, I think this threshold is likely optimal for what the grade is intended to represent.

Re: Definition of a FFR AA
 

I personally dont find 80g on a 2k notes file to be necessarily out of a subjective "good enough" range.

I'd like to know how you measure optimality though; do you have anything to say about the suggestion of using the current average notecount of ffr charts and setting a % relative to that ?

Re: Definition of a FFR AA
 

There's really nothing to lose by adding a SDG bar. The players like me who would want to farm the progress bar have a new goal, and the players that don't care won't be affected at all. The benefit a lot of people don't see, as in those who don't like to farm stuff like this anyways, is that the gap from 9 raw goods or less to AAA is relatively small. This change, for example, would give me incentive to grind "FMO's" for AA's when SDG's are very hard to come by at my skill level in that range. AA bar is better than an SDG bar for players like myself because it gives a slightly wider gap than between SDG and AAA and gives a new goal to aim for on charts that are still mostly manageable skill wise but not consistent enough yet to AAA. Also, percentage wise it should probably fall between 98-99% in my opinion, especially when SDGs are a bit over 99%. Also, the fact it would be a percentage is great for long difficult songs and helps balance the stamina required with the relative to the difficulty of the chart.

And no, I would not expect "AA's" to have any impact on a player's level or skill rating.

If this was implemented, I would probably start playing again. It's great for people who like to play the game the way I do (setting personal goals through in game metrics) and can just be ignored by those who don't care about it.

Re: Definition of a FFR AA
 

I dont think having AAs makes much sense when we already have SDGs, and i'd rather see an SDG bar added before we add an AA one. I think the main problem with AAs is that there seems to be a focus on % when thats not at all what FFR is about. Chart difficulty is judged on how hard a chart is to AAA and you are award points based on how many goods you are off from that AAA. If i AA a 22 on etterna, i'll get roughly the same score on both of them, however the AAA eq from AAing a really short 80 vs a really long 80 will be completely different. With the conversation of possibly removing FCs i think this moves in the opposite direction. instead of removing an arbitrary metric its adding another one. I just dont see AAs as a metric working under the current system unless they are decided by a flat raw good count.

Re: Definition of a FFR AA
 

Let me address a few points you make one by one. So this first statement implies that SDG and AA test the same thing, essentially. Otherwise if they didn't, it'd obviously make sense to have both.

Now as it's been said, SDG is extremely limited on many charts, being around at least 99.5x% raw score on most of them. Also, it's been shown that scores considered relatively "mash-y" could yield up to 95%-ish raw score. We're talking scores with over 100+ raw goods most of the time.

I think it's fair to say that there's a Wide margin between 10 raw goods and 100+ raw goods. Over in the FC thread, you can see the general sentiment from some players regarding "getting decent scores which may be a bit far from SDG still". An AA metric would effectively bridge that gap where players can also focus on that metric when it comes to charts just a bit outside of their comfort range, and it'd do so in a fun gamified way (being tracked and displayed).

Yeah that should be the case since it has less split opinion.

What is FFR about ? That sounds Extremely close-minded ngl, especially given these three facts:

1. Some people will enjoy that metric (as seen by the poll);
2. It affects in absolutely no way the way You, as someone who's not interested in it, play the game.
3. It's still a metric of performance, which seems to be something people focus super hard on (i.e. "don't put random metrics that aren't relevant to performance" etc)

That is a fact, yes. Has nothing to do with AA though, it's just how we compute AAA equivalency, which is FFR's main measure for skill (with skill rating).

You're making AA be tightly coupled with AAA equiv, which it doesn't have to (for the reasons mentionned above) and won't be. Also, I did go over chart structure in the OP; our AAA system is actually the one suffering the most in its current state, so that argument about a short/long 80 file is even less relevant given the fact that AA will Not be the main measure of skill FFR uses anyway.

I'd like to know how SDG isn't arbitrarily defined ? 10 raw goods on charts that can be full on stamina, where you either get 100 goods or AAA, VS 10 raw goods on a chart with a single hard spike, where you either get 15 goods or 5 goods.

I find it really selfish that Despite people explicitely saying they'd be motivated be the metric, others say it shouldn't be implemented because They find it arbitrary and pointless. I really fail to see how such a mindset can ever help making a game evolve and be less stagnant.

Would you care if it was implemented, and then based on the response and how much attention it gets from the users who like it, we keep/remove it ? Also the point about absolute vs relative raw good counts was made already: it alleviates the major problem I stated about chart structure.

Re: Definition of a FFR AA
 

The pushback on this proposal is a bit confusing to me. There are a lot of ways to play a 4key VSRG, as demonstrated by the variety of games that have risen to popularity these days. Over time these games are seeing more and more overlap in their playerbase and with that will come gradual changes in what players would like to see.

AA (or S-rank if youre from one of those games) is a pretty fundamental competency benchmark that simply has no equivalent in FFR at the moment. Neither FC nor SDG work as a viable substitute for this particular benchmark. And regardless of how one feels about FC or SDG, I don't think that has any strong impact on the AA benchmark and whether or not it'd be a useful inclusion.

Personally, I have no strong opinion about whether or not its based in raw goods or raw score, It doesn't matter to me how its reflected in the stats on the website (at least in the near future), and it doesn't particularly matter to me that an AA score isn't going to be particularly relevant to one's ranking- the important part is the utility and engagement it provides when playing stuff a bit outside of one's AAA / SDG range.

Re: Definition of a FFR AA
 

Maybe just have one bar that fills AAA / SDG / AA with different colors for each and then leave the FC bar as is (with FC* included)

Re: Definition of a FFR AA
 

I'm not as harshly against this as you seem to be implying. My main concern is that i dont think AAs as described compliment the systems FFR currently has in place. Grades are usually based on score and the score people "care" about is raw goods. In a world where the "goal" of FFR shifts away from flat goods, to a more % based scoring system, i'd agree that AAs are much more fitting there (and that shift might not be a bad thing). If graded scores fully make a comeback on FFR so be it, but i'm starting to get concerned that there are too many different flags, and i'm not sure i'd enjoy my game suddenly being filled with AA next to every song.

Re: Definition of a FFR AA
 

Maybe there could be a option in the engine to display AAs or not if it bothers some people

Re: Definition of a FFR AA
 

I dont understand this opinion, but I can't argue it