Go Back   Flash Flash Revolution > Flash Flash Revolution > FFR General Talk

View Poll Results: Should we add a AA metric to FFR ?
Yes, like how the OP described it 7 31.82%
Yes, but not exactly as described 5 22.73%
No 10 45.45%
Voters: 22. You may not vote on this poll

Reply
 
Thread Tools Display Modes
Old 08-16-2021, 10:23 AM   #1
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Definition of a FFR AA

This is a continuation thread based on the feedback from the FC implementation thread. Please go read that thread to know why this metric seems to be needed on FFR.

So, what is a AA ?

Basically a AA on other games serves the purpose of determining if you did "well enough" on a given chart, without necessarily being close to perfection on it. On Stepmania for example, this grade is obtained by getting 93.00% accuracy or better.

What is accuracy on FFR ?

Since FFR is frame based, it makes sense for its judgement values (Amazing, Perfect, Good, ...) to be defined by some frame window around the exact note timing. In any case, frames can always be converted to ms timing. That being said, the simplest way one could define accuracy on FFR is by comparing the obtained raw score on a chart with the AAA raw score on that same chart. This allows for any configuration of non-perfects to be a AA, as long the raw score is sufficiently high.

Other games like Etterna will consider even the slightest ms timing difference to compute the accuracy. In other words, a 100% is extremely rare and only achievable on super easy/short charts by very talented accuracy players. In the context of FFR, that would mean considering Amazings in the computation, where an Amazing would result in more points than a Perfect. Given the reputation of FFR to be relatively unstable at that scale, I think it's safe to say that Amazings should not make a difference.

When it comes to a specific % requirement, I believe it can be discussed mostly subjectively. I personally don't see why we couldn't use a round number like 90%. One interesting piece of data that could be useful is how much % can one get on average when just mashing charts. This would give a lower bound to the discussion. I don't have the time right now to compute that, but if someone could, that'd be very helpful.

What about chart structure ?

Chances are you havent asked yourself that question yet. Chart structure matters when you compare two charts, and one has a single difficulty spike whereas the other one is more balanced overall. This currently affects FFR in a major way since the AAA metrics that we use do not account for such chart structure, resulting in what is commonly known as "farm files". Well, the same logic applies to any AA metric, except that it introduces even more of these "farm files".

Although the AA metric would be even slightly more "broken" than our AAA metrics, it is far less of a big deal. The reason is that AA would never be FFR's main skill measure. Knowing that, there's no real need to fix AAA stuff before implementing AA. There's a non-0 chance that if/when difficulty is redefined in the future, AA will have to follow the changes, but that's probably the least of our concerns given the other implications that would come with a redefinition of difficulty.

What would AA give to users ?

Obviously, the first thing associated with AA's would be a completion metric just like the AAA's and FC's (and maybe SDG's eventually). Then, some Tokens and Skill Tokens could be made to use this new tracked grade.

________________________________

So what do you think of this suggestion, and do you have anything you'd like to see regarding this AA grade ?
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 10:47 AM   #2
Lights
owo
Head of Events Team
Event StaffFFR Simfile AuthorFFR Veteran
 
Lights's Avatar
 
Join Date: Jun 2020
Posts: 369
Default Re: Definition of a FFR AA

I think the most reasonable way to set an AA threshold in FFR is by using raw goods- similar to SDG. if AAA is 0 raw goods, and SDG is <10 raw goods, maybe somewhere between 30 and 50 raw goods? anything stricter than 30 is too close to SDG to really be valuable. 50 is a strong choice because at that threshold you're at least demonstrating some amount of control and competency on the patterns- maybe there was a bit of mashing on a difficulty spike and some bigger mistakes here and there, but you were doing something right for a good portion of the file.

(but if you think you can implement a competent accuracy% system for this, that'd be even better. i just think raw goods based fits well with the existing grades)

Last edited by Lights; 08-16-2021 at 10:49 AM..
Lights is offline   Reply With Quote
Old 08-16-2021, 10:51 AM   #3
Zeldagurlfan1
FFR Player
 
Zeldagurlfan1's Avatar
 
Join Date: Jan 2007
Location: Hyrule Castle
Age: 31
Posts: 333
Default Re: Definition of a FFR AA

Seems unnecessary tbh lol. Isn’t SDG basically a AA? And unless u want to score lower for tokens, that’s literally just letting the better scorers AAA until the end when they can get that I don’t play competitively often for myself, so outside looking in: it’s pointless aside from the clout of saying u got a AA. I’m know SDGs have their own reason for be excluded from this kind of scoreboard reasoning since it’s a kind of a 1/10 get if u can’t. How would u determine a AA? Getting more perfects than amazing? Would it be a percentage? Also if u do implement it, maybe make it like “perfects and amazing -only score”a limited to that same ‘1/10 get’ before the AAA. In the end if it does happen I’m looking forward to seeing my scores change and people get hyped since others will use it to work on their songs. If in the end the AAA metric is all that matters, I’d just kind of cut the idea unless u wanna hype competition. I also have no idea what metrics are like in these games coz I just go and play em, I mean as far as charting goes for this, I don’t see the relevance since for all I know judging is subjective too. Maybe it could help sort out more songs in the engine since there’s over 2k. Regardless it’s cute seeing everyone want to expand the game and I’m down to see your progress!!
Zeldagurlfan1 is offline   Reply With Quote
Old 08-16-2021, 10:51 AM   #4
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

Quote:
Originally Posted by Lights View Post
I think the most reasonable way to set an AA threshold in FFR is by using raw goods- similar to SDG. if AAA is 0 raw goods, and SDG is <10 raw goods, maybe somewhere between 30 and 50 raw goods? anything stricter than 30 is too close to SDG to really be valuable. 50 is a strong choice because at that threshold you're at least demonstrating some amount of control and competency on the patterns- maybe there was a bit of mashing on a difficulty spike and some bigger mistakes here and there, but you were doing something right for a good portion of the file.

Why an absolute raw good count vs a relative one though ? To me it seems like absolute count would simply be too similar to the other flags we use (Blackflag, Booflag, SDG), whereas a relative count would bring in some variety to FFR while also catering to the players from other games.
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 10:57 AM   #5
benny58624
FFR Veteran
FFR Veteran
 
benny58624's Avatar
 
Join Date: May 2014
Age: 28
Posts: 351
Default Re: Definition of a FFR AA

A value of 95% of the maximum possible raw score would sound good. It's a nice round number, high enough to show you managed to play the file decently, and low enough to be significantly easier than SDG. A value of 90% would be too low: looking at my past scores I've already mashed my way through some difficult files, getting 90%.
benny58624 is offline   Reply With Quote
Old 08-16-2021, 10:58 AM   #6
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

Quote:
Originally Posted by benny58624 View Post
A value of 95% of the maximum possible raw score would sound good. It's a nice round number, high enough to show you managed to play the file decently, and low enough to be significantly easier than SDG. A value of 90% would be too low: looking at my past scores I've already mashed my way through some difficult files, getting 90%.
Oh nice, can you please give some examples of those scores in here ? In fact, after some discussion on the % on discord, 90% definitely IS far too low for most charts. When I get access to my laptop I will compute the average raw score % that a SDG represents on all files, and feom there we could adjust the AA % to be something lower.

Last edited by xXOpkillerXx; 08-16-2021 at 11:15 AM..
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 11:32 AM   #7
benny58624
FFR Veteran
FFR Veteran
 
benny58624's Avatar
 
Join Date: May 2014
Age: 28
Posts: 351
Default Re: Definition of a FFR AA

Quote:
Originally Posted by xXOpkillerXx View Post
Oh nice, can you please give some examples of those scores in here ? In fact, after some discussion on the % on discord, 90% definitely IS far too low for most charts. When I get access to my laptop I will compute the average raw score % that a SDG represents on all files, and feom there we could adjust the AA % to be something lower.
The tier points page gives a nice comparison of obtained raw score and maximum possible raw score for a bunch of files. I see I've managed to get a 93885/104000 = 90.3% score on System Doctor. Not sure when I got it, but to compare: I'm mid-high D4 now, and should definitely NOT be able to get AA on a difficulty 101 file.

Other examples:
92.8% Revolutionary Etude
90.3% A Quick Death
92.7% Grist
91.0% Magical 8bit Tour
91.0% Schmollbluk
... and more.

For a 1000 notes file, a SDG equals a score of 99.51+%.

Last edited by benny58624; 08-16-2021 at 11:37 AM..
benny58624 is offline   Reply With Quote
Old 08-16-2021, 11:42 AM   #8
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

Quote:
Originally Posted by benny58624 View Post
The tier points page gives a nice comparison of obtained raw score and maximum possible raw score for a bunch of files. I see I've managed to get a 93885/104000 = 90.3% score on System Doctor. Not sure when I got it, but to compare: I'm mid-high D4 now, and should definitely NOT be able to get AA on a difficulty 101 file.

Other examples:
92.8% Revolutionary Etude
90.3% A Quick Death
92.7% Grist
91.0% Magical 8bit Tour
91.0% Schmollbluk
... and more
Great, thanks for the info ! A more accurate estimate has been made at around 97%, considering an SDG on a 1500 notes chart is 99.333%. On that same notecount, a 97% would mean 90 raw goods, and on a chart with 3000 notes it would be 180 raw goods.
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 01:35 PM   #9
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

My bad if the raw goods counts in my examples are wrong, typing quickly on phone.

The idea remains that the % for AA should be agreed on w.r.t. the average notecount to be as representative as possible.
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 01:43 PM   #10
Dynam0
The Dominator
D7 Elite KeysmasherFFR Veteran
 
Dynam0's Avatar
 
Join Date: Sep 2005
Location: North Bay, ON
Age: 34
Posts: 8,987
Default Re: Definition of a FFR AA

Quote:
Originally Posted by xXOpkillerXx View Post
My bad if the raw goods counts in my examples are wrong, typing quickly on phone.

The idea remains that the % for AA should be agreed on w.r.t. the average notecount to be as representative as possible.

I can't see a better way than this...though you will still have those anomalous files that have a huge skill gap to go from AA -> SDG (System Doctor is a great example, very farmable as it is for skill rating lol) and then other files that you can improve drastically on with a small skill increase (stamina intensive and marathon files)

I don't see anything wrong with the idea though since all we're doing is counting the AA's and not assigning ratings to them based on difficulty
Dynam0 is offline   Reply With Quote
Old 08-16-2021, 01:50 PM   #11
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

Quote:
Originally Posted by Dynam0 View Post
I can't see a better way than this...though you will still have those anomalous files that have a huge skill gap to go from AA -> SDG (System Doctor is a great example, very farmable as it is for skill rating lol) and then other files that you can improve drastically on with a small skill increase (stamina intensive and marathon files)

I don't see anything wrong with the idea though since all we're doing is counting the AA's and not assigning ratings to them based on difficulty
Yeah the flaws of our AAA system shouldnt be applied to this since it would purely be a rough measure of "did you get a somewhat decent score". It still suffers from the same-ish chart structure I mentionned in the OP yes, but I dont think it's important given the goal of the metric.
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 02:09 PM   #12
WirryWoo
Forever Derbyless
Retired StaffFFR Simfile Author
 
WirryWoo's Avatar
 
Join Date: Aug 2020
Age: 32
Posts: 240
Default Re: Definition of a FFR AA

If the primary goal of introducing AA is to replace FC counts, I disagree. In other rhythm game communities, AA is simply just a grade received on your performance of the file. This can cause more confusion for new Etterna players thinking "wtf why is it so easy to get an AA on FFR lmfao?" If you really want to introduce AA, you might as well start introducing other grades like A, B, C, etc. in this definition as well.

However, any hypothetical AA, A, B, C, D, etc. is not contributive to your skill rating. It's only just letters designed to unofficially and unnecessarily validate your own skills while playing this game. I say unofficially because these grades are not factored into the skill ratings computation and I say unnecessarily because your scores should be enough to make you feel proud.

If the skill ratings formula is revamped to somehow incorporate these grades, then my vote would have switched to "Yes". Right now, the AA proposal just seems like another one of millions of random stats we currently have on this website lmfao.
__________________
                   
WirryWoo is offline   Reply With Quote
Old 08-16-2021, 03:05 PM   #13
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

Quote:
Originally Posted by WirryWoo View Post
If the primary goal of introducing AA is to replace FC counts
No it's not. It came from the FC discussion, but it's not related to FC at all.

Quote:
In other rhythm game communities, AA is simply just a grade received on your performance of the file. This can cause more confusion for new Etterna players thinking "wtf why is it so easy to get an AA on FFR lmfao?" If you really want to introduce AA, you might as well start introducing other grades like A, B, C, etc. in this definition as well.
If we start doing everything exactly like other games do, what's the point of being a different game ? Sure there's common sense like don't make "AA" mean you passed a song, but the point still holds.

Quote:
However, any hypothetical AA, A, B, C, D, etc. is not contributive to your skill rating. It's only just letters designed to unofficially and unnecessarily validate your own skills while playing this game. I say unofficially because these grades are not factored into the skill ratings computation and I say unnecessarily because your scores should be enough to make you feel proud.
Firstly it's not for you to decide how others perceive/gauge their performance; we have a main metric in FFR which is skill rating, outside of that can live other metrics to diversify the game.

Secondly, implying it's necessairly a pride thing is close minded: why can't people just have Fun trying to track how many files they can play decently on with a structured, automatic metric ?

Quote:
If the skill ratings formula is revamped to somehow incorporate these grades, then my vote would have switched to "Yes". Right now, the AA proposal just seems like another one of millions of random stats we currently have on this website lmfao.
Ok, your point is simply "I think we have enough secondary objectives". That's fine, but it's entirely subjective. If you dislike seeing more of these which arent mixed with the main metric, you're free to say so. Otherwise, there's no point in forbidding others to have this metric they find interesting/fun for whatever reason.
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 03:51 PM   #14
123kappa3
FFR Veteran
FFR Veteran
 
123kappa3's Avatar
 
Join Date: Nov 2016
Age: 28
Posts: 69
Default Re: Definition of a FFR AA

I dont know what this adds to ffr. I am not very understanding of everything, but ffr is a aaa game, you goal is to get a aaa, not a aa, so adding a aa type of score feels like its not ffr.
123kappa3 is offline   Reply With Quote
Old 08-16-2021, 03:58 PM   #15
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

Quote:
Originally Posted by 123kappa3 View Post
I dont know what this adds to ffr. I am not very understanding of everything, but ffr is a aaa game, you goal is to get a aaa, not a aa, so adding a aa type of score feels like its not ffr.
Just like tokens, skill tokens, tier points, badges or any other secondary element of FFR that is not the main skill measure, AA would give another optional metric to track/work towards.
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 04:18 PM   #16
123kappa3
FFR Veteran
FFR Veteran
 
123kappa3's Avatar
 
Join Date: Nov 2016
Age: 28
Posts: 69
Default Re: Definition of a FFR AA

Quote:
Originally Posted by xXOpkillerXx View Post
Just like tokens, skill tokens, tier points, badges or any other secondary element of FFR that is not the main skill measure, AA would give another optional metric to track/work towards.
I guess, its fine, but it feels like you could just calculate it on your own? idk why getting a aa is that important for ffr.
123kappa3 is offline   Reply With Quote
Old 08-16-2021, 04:21 PM   #17
Fantasticone
D7 Elite Keymasher
D7 Elite KeysmasherFFR Veteran
 
Fantasticone's Avatar
 
Join Date: Aug 2006
Age: 33
Posts: 6,003
Default Re: Definition of a FFR AA

2% raw goods / total notes in song
__________________

Last edited by Fantasticone; 08-16-2021 at 04:50 PM..
Fantasticone is offline   Reply With Quote
Old 08-16-2021, 04:24 PM   #18
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

Quote:
Originally Posted by 123kappa3 View Post
I guess, its fine, but it feels like you could just calculate it on your own? idk why getting a aa is that important for ffr.
You could also count your AAAs by going through the list in level ranks of your profile.

It's not -important-, it's a suggestion to implement a feature which may incentivize some people to work towards instead of focusing on AAAs. Nobody asserted that it's a necessity nor that it will be relevant for all players (just like the other optional things I mentionned that FFR has).
xXOpkillerXx is offline   Reply With Quote
Old 08-16-2021, 04:47 PM   #19
benny58624
FFR Veteran
FFR Veteran
 
benny58624's Avatar
 
Join Date: May 2014
Age: 28
Posts: 351
Default Re: Definition of a FFR AA

Quote:
Originally Posted by xXOpkillerXx View Post
If we start doing everything exactly like other games do, what's the point of being a different game ? Sure there's common sense like don't make "AA" mean you passed a song, but the point still holds.
Fun fact: FFR did show these AA/A/B/C/D/E rates already on the results screen, but then it got removed. I don't remember how it exactly worked, it likely went off combo score. They weren't tracked anywhere else AFAIK.

benny58624 is offline   Reply With Quote
Old 08-16-2021, 04:59 PM   #20
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Definition of a FFR AA

Quote:
Originally Posted by benny58624 View Post
Fun fact: FFR did show these AA/A/B/C/D/E rates already on the results screen, but then it got removed. I don't remember how it exactly worked, it likely went off combo score. They weren't tracked anywhere else AFAIK.

Yep, I'm aware of that

They were only part of the game engine and not tracked anywhere, so it seems very normal that it never was interesting enough for anyone to care about.

Reason why I only suggested some kind of AA equivalent is twofold:
1. On FFR, as we've discussed in this thread, you can relatively easily mash your way to a % as high as 95%-ish, because of how our judgements work. Therefore anything worse than that AA metric would converge to the same required effort to obtain for even less significance than the AA compared to AAA.

2. In some other games, AA is a very common metric for players to determine if they did decent on a file. Very rarely do we see players go for the lower grades.
xXOpkillerXx is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -5. The time now is 06:56 AM.


Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright FlashFlashRevolution