Go Back   Flash Flash Revolution > Flash Flash Revolution > FFR General Talk

Reply
 
Thread Tools Display Modes
Old 07-7-2018, 01:45 PM   #41
RenegadeLucien
FFR Veteran
Skill Rating Designer
Retired StaffFFR Veteran
 
RenegadeLucien's Avatar
 
Join Date: Jan 2016
Age: 27
Posts: 282
Default Re: Entropy Gain for per-receptor NPS

I'd need to experiment with it to get a definitive answer. I can see the value in having something like that, but it would be difficult to separate actual spikes/bursts from just natural variance in patterns (take a staircase for example: there are gaps of 5 notes between every left arrow, but only 1 between (some) down or up arrows, so the down/up arrows look much harder than the left/right arrows, and this could produce odd results for a difficulty change rate value. Would probably have to look at average difficulty over a short period of notes and use that to determine the difficulty change rate.
__________________


RenegadeLucien is offline   Reply With Quote
Old 07-7-2018, 04:26 PM   #42
AutotelicBrown
Under the scarlet moon
FFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
AutotelicBrown's Avatar
 
Join Date: Jan 2014
Age: 31
Posts: 921
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by xXOpkillerXx View Post
What do you mean by "if you are going to use those as ground truth" ? I said I'm going the unsupervised way, there is no ground truth in that afaik. I plan on doing any estimation based on difficulties, not scores. Sorry if I misunderstood your point.
Sorry, I misread your original statement I quoted before. You can ignore that part.
AutotelicBrown is offline   Reply With Quote
Old 07-7-2018, 04:57 PM   #43
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by RenegadeLucien View Post
I'd need to experiment with it to get a definitive answer. I can see the value in having something like that, but it would be difficult to separate actual spikes/bursts from just natural variance in patterns (take a staircase for example: there are gaps of 5 notes between every left arrow, but only 1 between (some) down or up arrows, so the down/up arrows look much harder than the left/right arrows, and this could produce odd results for a difficulty change rate value. Would probably have to look at average difficulty over a short period of notes and use that to determine the difficulty change rate.
I'm not sure, why would you want to seperate those ? It would help me understand if you defined spikes as opposed to natural variance. Lets say we use the distance metric and focus on the up arrow of a long staircase: that receptor is now essentially receiving minijacks of 2 notes (seperated by 1 note on right) every 4 notes. The rate of change can then be computed like this:

up, ,up, , , ,up, ,up
__, ,0 , , , ,-2, ,2

vs

ri, , , ,ri, , , ,ri, , , ,ri
_, , , ,0, , , ,0, , , ,0

(changes between 0 and 1 have been normalized to the opposite of their inverse: 0.5 => 2 => -2)

Takes a minimum of 3 notes to have a variation in distance. While it's true that the average is the same (0), you could maybe take the range between the minimum negative value (biggest deceleration) and the maximum positive value (biggest acceleration).

Deceleration doesn't affect difficulty, don't forget that this is a per-receptor metric. A file starts at 0 difficulty with 0 notes. If you put a jack at x speed, then after a few notes its speed changes to x/2, the only problem is going from 0 speed to x speed, not from x to x/2. Gradual acceleration/deceleration aren't considered in this but you can get a primitive for it using this same concept. So, for the example of the staircase, if we discard the negative values, we get a max range of 2 on up and down, and a max range of 0 on left and right. And you dont aggregate those in any way because the min/max on each receptor is important.

Does that cover the type of example you had in mind, Renegade ?
xXOpkillerXx is offline   Reply With Quote
Old 07-7-2018, 05:23 PM   #44
RenegadeLucien
FFR Veteran
Skill Rating Designer
Retired StaffFFR Veteran
 
RenegadeLucien's Avatar
 
Join Date: Jan 2016
Age: 27
Posts: 282
Default Re: Entropy Gain for per-receptor NPS

So, on spikes vs natural variance: what I mean by "spike", at least in the context of saying that my old algorithms would rate spiky files way too high, was files such as ABCDEath or TTE which have one disproportionately note-heavy section that overshadows everything else in the file. When I say "natural variance", I mean that some arrows in a long pattern like a stream, jumpstream, or staircase will be harder to hit than others.

What I'm trying to avoid is to see a staircase, get a max range of 2 on up or down as you described, and falsely claim that the staircase is a spike when in reality, it's just a staircase. Whatever metric that is used to determine the rate would have to be able to tell the difference.
__________________


RenegadeLucien is offline   Reply With Quote
Old 07-7-2018, 06:55 PM   #45
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by RenegadeLucien View Post
So, on spikes vs natural variance: what I mean by "spike", at least in the context of saying that my old algorithms would rate spiky files way too high, was files such as ABCDEath or TTE which have one disproportionately note-heavy section that overshadows everything else in the file. When I say "natural variance", I mean that some arrows in a long pattern like a stream, jumpstream, or staircase will be harder to hit than others.

What I'm trying to avoid is to see a staircase, get a max range of 2 on up or down as you described, and falsely claim that the staircase is a spike when in reality, it's just a staircase. Whatever metric that is used to determine the rate would have to be able to tell the difference.
There are a few different points in this so I'll try to dissect it as clearly as possible.

You have to have some mathematical definition of your concepts if not used for visualisation only. For example, a spike would be a sudden high density x of notes, at least to my understanding of your description. In a more formal way you could say it's any section with high acceleration (lets use a trivial number like 4). Also btw my metric isn't totally correct for another reason, I'll post a fix to it.

So you then have a trivial definition of a spike. With that, you want to avoid cases where the spike is short (i.e. in a staircase, the two ups or two downs) and constant (the staircase goes on for some time like 2 measures). The reason it's trivial is because first of all there is a trivial threshold to set and also because the length of said spike is not well bounded.

You mention TTE. Take TTE's fastest spot (a rolly burst like 123412341234) and remove everything before it. The acceleration from nothing to that is equal for each receptor, so min = max = x. Now take a staircase 123432123432 with the distance between two up arrows being equal in this and the roll (from a per-receptor perspective, that is most definitely fair). From nothing to it, 2*min = max = x. It would seem that both are identical, however for the comparison to hold the total nps of the spike will be lower on the staircase than on the roll (the amount of notes bewteen the fastest consecutive notes per-receptor being 1 for the staircase and 3 for the roll). Therefore, a distinction Should be made naturally but the spikiness (again, per-receptor!) will be the same according to the trivial definition.

EDIT:
Just to be extra clear, I'll point out that what you refer to as a spike as we all know it is easily defined when using all notes (not per-receptor). There's a quick increase and decrease in the nps of the section and that's it. That metric can be useful, but it's not what I was explaining/arguing in the previous few posts.

Last edited by xXOpkillerXx; 07-7-2018 at 07:01 PM..
xXOpkillerXx is offline   Reply With Quote
Old 07-7-2018, 10:14 PM   #46
RenegadeLucien
FFR Veteran
Skill Rating Designer
Retired StaffFFR Veteran
 
RenegadeLucien's Avatar
 
Join Date: Jan 2016
Age: 27
Posts: 282
Default Re: Entropy Gain for per-receptor NPS

Yeah I think we're talking about totally different concepts here. Per-receptor spikiness isn't something I ever really considered in my algorithm, at least not beyond "this note is really close to the last note for this receptor, therefore it should have a high value".

I can't think of any files off the top of my head where per-receptor spikiness plays a major factor in the difficulty of the file, so I can't judge how well the simple "this note is close" factor covers it. I do think such a metric would be valuable to have.
__________________


RenegadeLucien is offline   Reply With Quote
Old 07-8-2018, 11:16 AM   #47
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by RenegadeLucien View Post
Yeah I think we're talking about totally different concepts here. Per-receptor spikiness isn't something I ever really considered in my algorithm, at least not beyond "this note is really close to the last note for this receptor, therefore it should have a high value".

I can't think of any files off the top of my head where per-receptor spikiness plays a major factor in the difficulty of the file, so I can't judge how well the simple "this note is close" factor covers it. I do think such a metric would be valuable to have.
Think Crowdpleaser, Death Piano ending, RAN's trilly bursts section or even party4u v1's 0-framers for more intense examples. Any jacks (per-receptor) faster than their surrounding notes basically. If you take total nps only, Death Piano's ending roll and trill are the same difficulty but obviously it's not the case. Per-receptor nps will clearly make the difference and rate the trill much higher than the roll.
xXOpkillerXx is offline   Reply With Quote
Old 07-8-2018, 11:36 AM   #48
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

Here's something I am wondering about per-hand stuff.

Is it safe to assume that any section with a high nps (x) on {3} and a lower nps on {4} is Harder than having x nps on both {3} and {4} ? Any counterexample is welcome.

More visually, I'm thinking that [34]4[34]4 is always harder than [34][34][34][34]. But only per-hand, so the same wouldn't apply with combinations of receptors like {2} and {3}, or {2} and {4}, etc. And by always I mean no matter what is before it, after it, and what's going on on the other receptors.

EDIT:
I will even go as far as claiming that if x is the nps on {3} and y is the nps on {4}, the peak of that per-hand difficulty is reached when x = 2y or 2x = y. When you lower small nps, you get things like [34]44[34]44[34]44, and when you raise it, you get [34][34]4[34][34]4, both of which I would argue are objectively easier than [34]4[34]4[34]4.

Last edited by xXOpkillerXx; 07-8-2018 at 12:29 PM.. Reason: oops meant easier in last sentence
xXOpkillerXx is offline   Reply With Quote
Old 07-8-2018, 12:42 PM   #49
RenegadeLucien
FFR Veteran
Skill Rating Designer
Retired StaffFFR Veteran
 
RenegadeLucien's Avatar
 
Join Date: Jan 2016
Age: 27
Posts: 282
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by xXOpkillerXx View Post
If you take total nps only, Death Piano's ending roll and trill are the same difficulty but obviously it's not the case. Per-receptor nps will clearly make the difference and rate the trill much higher than the roll.
I wasn't talking about per-receptor NPS. Of course that is important, and my algorithm already covers it since it measures the distance between notes on the same receptor (in fact, in its current state, it freaks out at the DP megatrill and rates it higher than Undici). I was talking about the per-receptor spikiness that you had been describing in your last few posts. I don't know if there are any files where per-receptor spikiness add any difficulty that isn't accounted for by per-receptor NPS or distance between two notes on the same receptor.

Quote:
Originally Posted by xXOpkillerXx
Is it safe to assume that any section with a high nps (x) on {3} and a lower nps on {4} is Harder than having x nps on both {3} and {4} ? Any counterexample is welcome.
Mostly, as long as the nps on 4 is high enough. I don't really think, say, a long 3 jack with one random {34} jump (ex. AIM Anthem) in the middle is harder than a long {34} jack.
__________________


RenegadeLucien is offline   Reply With Quote
Old 07-8-2018, 01:09 PM   #50
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by RenegadeLucien View Post
I wasn't talking about per-receptor NPS. Of course that is important, and my algorithm already covers it since it measures the distance between notes on the same receptor (in fact, in its current state, it freaks out at the DP megatrill and rates it higher than Undici). I was talking about the per-receptor spikiness that you had been describing in your last few posts. I don't know if there are any files where per-receptor spikiness add any difficulty that isn't accounted for by per-receptor NPS or distance between two notes on the same receptor.



Mostly, as long as the nps on 4 is high enough. I don't really think, say, a long 3 jack with one random {34} jump (ex. AIM Anthem) in the middle is harder than a long {34} jack.
Ok I get what you mean with that first paragraph. Indeed, there might not be a need for anything more than nps for per-receptor metrics. I will try to find any example where it would matter but on top of my head I dont see any either.

How high does your algo rate DP compared to Undici ? Those are really hard files and none have been AAA'd yet Right now, they're only 2 points apart from each other, and I would Not consider the opposite to be an error because it's only a single file (it's better to look at results as a whole first and then understand the difference between particular files, so not having your complete results, I can only assume things).

As for you AIM example, it would make no sense to put a long jack and a long jack with 1 jump in it at the same exact difficulty on a real numbers scale. The one with the jump Has to be harder, even if it's by a very small amount.
xXOpkillerXx is offline   Reply With Quote
Old 07-8-2018, 01:11 PM   #51
leonid
I am leonid
Retired StaffFFR Simfile AuthorFFR Music ProducerD7 Elite KeysmasherFFR Veteran
 
leonid's Avatar
 
Join Date: Oct 2008
Location: MOUNTAIN VIEW
Age: 34
Posts: 8,080
Default Re: Entropy Gain for per-receptor NPS

Does it also address the fact that difficulty differs based on what your goal is? Charts can be trivial to AA but impossible to AAA, or there's some stupid minefield that makes it really hard to pass but once you survive it's a guaranteed AA, etc
__________________



Proud member of Team No

Last edited by leonid; 07-8-2018 at 01:12 PM..
leonid is offline   Reply With Quote
Old 07-8-2018, 01:25 PM   #52
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by leonid View Post
Does it also address the fact that difficulty differs based on what your goal is? Charts can be trivial to AA but impossible to AAA, or there's some stupid minefield that makes it really hard to pass but once you survive it's a guaranteed AA, etc
Right now I'm focusing solely on FFR. Since we use AAA equivalency, most of the difficulty will come from max values and length/stamina factors.

EDIT:
@leonid: stepmania is different obviously. What you describe as difficulty to AA, AAA, pass, are all very distinct values that may have a similar computing process but would have their own specific primitives. You can't possibly have a single metric for overall difficulty when your definition of difficulty is an undefined combination of 3 distinct aspects, otherwise you end up with obviously biaised results that are very hard to interpret. A fair comparison can be made with Etterna's calculator: if overall difficulty is some aggregate (like avg or weighted avg) of the per-pattern difficulties (jack, stream, js, etc), then it's not a surprise that they haveso many files to ban from leaderboards.

Last edited by xXOpkillerXx; 07-8-2018 at 01:50 PM..
xXOpkillerXx is offline   Reply With Quote
Old 07-8-2018, 01:52 PM   #53
RenegadeLucien
FFR Veteran
Skill Rating Designer
Retired StaffFFR Veteran
 
RenegadeLucien's Avatar
 
Join Date: Jan 2016
Age: 27
Posts: 282
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by xXOpkillerXx View Post
How high does your algo rate DP compared to Undici ? Those are really hard files and none have been AAA'd yet Right now, they're only 2 points apart from each other, and I would Not consider the opposite to be an error because it's only a single file (it's better to look at results as a whole first and then understand the difference between particular files, so not having your complete results, I can only assume things).
Way higher. I haven't scaled it to match the 1-120 (or 1-99) FFR scale, but Undici is given a value of 49.5 (for some comparisons, RATO is 46.4, Magical 8bit Tour is 41.2, La Camp is 39.4). DP is given 62.7.

Quote:
As for you AIM example, it would make no sense to put a long jack and a long jack with 1 jump in it at the same exact difficulty on a real numbers scale. The one with the jump Has to be harder, even if it's by a very small amount.
Yes, the 3 jack with the one 4 jump is harder than a 3 jack. But I question it being harder than a {34} jack. I don't know about you, but I can do faster 3 jacks than I can do {34} jacks.
__________________


RenegadeLucien is offline   Reply With Quote
Old 07-8-2018, 02:00 PM   #54
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by RenegadeLucien View Post
Way higher. I haven't scaled it to match the 1-120 (or 1-99) FFR scale, but Undici is given a value of 49.5 (for some comparisons, RATO is 46.4, Magical 8bit Tour is 41.2, La Camp is 39.4). DP is given 62.7.



Yes, the 3 jack with the one 4 jump is harder than a 3 jack. But I question it being harder than a {34} jack. I don't know about you, but I can do faster 3 jacks than I can do {34} jacks.
Ok yes that is an odd result haha.

As for the jacks, the difficulty of perhand would be symmetric on both sides of of the 2:1 ratio, but the nps primitive would naturally make a [34] jumpjack harder than a 3 jack with a single [34] in it.
xXOpkillerXx is offline   Reply With Quote
Old 07-8-2018, 02:47 PM   #55
blanky!
Banned
D7 Elite KeysmasherFFR Veteran
 
Join Date: Jan 2015
Posts: 164
Send a message via Skype™ to blanky!
Default Re: Entropy Gain for per-receptor NPS

I don't like difficulty being "one value".
It should vary in magnitude throughout the song, and be different kinds of difficulty.

Like, how do scores change if you're only slightly less good at hitting something than another player. Difficulty might not be that well, but if it means the difference between AAA'ing and good-rushing a difficult jumpstream, then the scores are highly sensitive to skill. Maybe that's a good measure? Change in score vs. change in skill in a certain direction? Dunno. Thoughts aren't fleshed out at all. Just food for thought.
blanky! is offline   Reply With Quote
Old 07-8-2018, 04:08 PM   #56
RenegadeLucien
FFR Veteran
Skill Rating Designer
Retired StaffFFR Veteran
 
RenegadeLucien's Avatar
 
Join Date: Jan 2016
Age: 27
Posts: 282
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by blanky! View Post
Like, how do scores change if you're only slightly less good at hitting something than another player. Difficulty might not be that well, but if it means the difference between AAA'ing and good-rushing a difficult jumpstream, then the scores are highly sensitive to skill. Maybe that's a good measure? Change in score vs. change in skill in a certain direction? Dunno. Thoughts aren't fleshed out at all. Just food for thought.
Thing is, for FFR specifically, this is a moot point because FFR difficulties are based strictly on the difficulty to AAA a song. It doesn't matter if a song is more prone to blackflags or 15g scores for people who are close to AAA'ing it, what matters is that the rating given to the song is higher then the rating that these players can AAA. Though I'd question that a player who consistently good-rushes a section of a song and ends up with 15g is close to AAA'ing it at all.
__________________


RenegadeLucien is offline   Reply With Quote
Old 07-9-2018, 02:53 PM   #57
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

Quote:
Originally Posted by blanky! View Post
I don't like difficulty being "one value".
It should vary in magnitude throughout the song, and be different kinds of difficulty.

Like, how do scores change if you're only slightly less good at hitting something than another player. Difficulty might not be that well, but if it means the difference between AAA'ing and good-rushing a difficult jumpstream, then the scores are highly sensitive to skill. Maybe that's a good measure? Change in score vs. change in skill in a certain direction? Dunno. Thoughts aren't fleshed out at all. Just food for thought.
Could you elaborate on that bold part please ? I'm not sure I understand what you're asking for.

Also here's another idea, but I haven't found a way to make it fully non-trivial yet:
I can take the nps at every frame of a file (kinda like the nps generator), and figure out how long the file stays around its max nps. The only problem with that is I can't just put a random threshold like "time during which nps is at most 2nps away from max nps" or "time during which nps is higher than 95% of the max nps". Thoughts ?
xXOpkillerXx is offline   Reply With Quote
Old 08-22-2018, 09:36 PM   #58
MinaciousGrace
FFR Player
D7 Elite Keysmasher
 
MinaciousGrace's Avatar
 
Join Date: Dec 2007
Location: nima
Posts: 4,278
Default Re: Entropy Gain for per-receptor NPS

so did you get far enough to realize you have no idea what you're doing yet or did you just spout a bunch of bullshit and then do nothing

i swear you people that think everything can be solved with machine learning are worse than the people who think blockchain makes everything better
MinaciousGrace is offline   Reply With Quote
Old 08-22-2018, 09:56 PM   #59
xXOpkillerXx
Forever OP
Simfile JudgeFFR Simfile AuthorD8 Godly KeysmasherFFR Veteran
 
xXOpkillerXx's Avatar
 
Join Date: Dec 2008
Location: Canada,Quebec
Age: 28
Posts: 4,168
Default Re: Entropy Gain for per-receptor NPS

I didn't want to be pushy towards prawn because he seemed very busy already. I talked with him and for tests on the whole song db I need to ask him everytime. I have converted a few sm packs for basic tests but since those arent rated like ffr, I couldnt just accept/discard results (there also were some sketchy numbers with stuff like Beyond Bludgeonned, Big Black and Little piece of Heaven, stuff that extrapolation should still rate correctly). I still have formulas to try yet, but now I'm on vacation and not focusing on ffr at all. Will most likely get back to it when my semester starts (early september).

You seem to be in quite the hurry to see results for an arrogant ass. I think even if I manage to get good results I'll hide them from you.
xXOpkillerXx is offline   Reply With Quote
Old 08-23-2018, 01:47 PM   #60
MinaciousGrace
FFR Player
D7 Elite Keysmasher
 
MinaciousGrace's Avatar
 
Join Date: Dec 2007
Location: nima
Posts: 4,278
Default Re: Entropy Gain for per-receptor NPS

so basically you're in total denial still
MinaciousGrace is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -5. The time now is 01:15 PM.


Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright FlashFlashRevolution