Old 08-17-2014, 05:54 AM   #1
ilikexd
Retired StaffFFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
Join Date: Apr 2006
Posts: 3,205
Default Rework the SAP system

The SAP system is a clear necessity in the FFR submission process to assure quality control. But I think there is a problem in how points are collected and in the submission cap tiers.

The idea behind submission caps, as I understand it, is as follows: Everyone should have the opportunity to submit simfiles to FFR, but the judgement process is a long and laborious one, so in order to save time and effort for simfile judges and batch organizers, simfile artists are restricted to a certain number of simfiles they can send per period. Naturally, this compels them to only send simfiles that they think are their best and have a chance at being accepted, thus raising the average quality of all submissions. When a simfile artist gets many simfiles accepted, his or her submission cap is raised based on the idea that the simfiles being produced are of a higher quality and less likely to be rejected. The problem is that the whole concept of a higher cap for the purpose of having a higher ratio of acceptance-worthy to rejection-prone simfiles in a batch isn't being executed optimally.

I do not think there should be an arbitrary limit of five at 40+ points. I can more sympathize with the limit of five back when batches would become full and close, but now that the batch is permanently open, I don't. If the simfile artist has truly proven him or herself by acquiring so many points, there should be even further cap tiers, as long as the points are a reliable measure.

That brings me to another point: I don't think they are that reliable. While having a lot of SAP is indeed a measure of simfiling capability, it becomes pretty inaccurate for a few reasons. If one in every three simfiles a simfile artist submits is accepted, he or she will be getting SAP over time, and his or her submission cap will raise indefinitely. But the intended purpose of the SAP system assuring quality control is not fulfilled; the opposite happens (this is an extreme theoretical example to demonstrate a point and doesn't happen to this extent in practice). Instead of SAP being granted for every file acceptance, even when those acceptances are accompanied by the same number or more rejections in a batch, SAP should be granted based on overall performance inside a single batch. One acceptance accompanied by more than one rejection is not a positive performance and should not yield +SAP.

There is also no real way to lose SAP. Technically SAP is lost when receiving 1.5/10 or lower average on a simfile. But as far as I know this has never happened to any simfile artist who has any charts in game, so this measure is entirely useless. But it seems that based on its existence, somewhere along the line there was some intention to implement a system that punishes negative performance. Now to be sincere the actual prospect of having SAP reduced from a negative performance is pretty harsh, but as I wrote earlier I think SAP should be granted based on the overall/net performance in a batch, so that would mean considering both positive and negative performance within the same batch, which would not be a punishment/deduction in itself but would be so indirectly and would be an even better motivator for sending one's best simfiles and only one's best simfiles, and limit any frivolous submissions.

Last edited by ilikexd; 08-17-2014 at 05:56 AM..
ilikexd is offline   Reply With Quote
Old 08-17-2014, 08:21 AM   #2
M0nkeyz
Simfile Judge
Simfile JudgeDifficulty ConsultantFFR Simfile AuthorFFR Veteran
 
M0nkeyz's Avatar
 
Join Date: May 2013
Posts: 476
Default Re: Rework the SAP system

If all quality step artists get like 10 SAP, how much room is there for new artists if all the good step artists inevitably raise the bar. You can't really expect the same results from a stepartist who has been stepping for 5 years, and a stepartist who maybe has been stepping for only 3 months. But what happens then? Do you accept all 10 files? This will inevitably make the judges more strict on accepting files making it even harder for the new guys.
M0nkeyz is offline   Reply With Quote
Old 08-17-2014, 09:08 AM   #3
TC_Halogen
Rhythm game specialist.
Retired StaffFFR Simfile AuthorFFR Music ProducerD8 Godly KeysmasherFFR Veteran
 
TC_Halogen's Avatar
 
Join Date: Feb 2008
Location: Bel Air, Maryland
Age: 32
Posts: 19,376
Send a message via AIM to TC_Halogen Send a message via Skype™ to TC_Halogen
Default Re: Rework the SAP system

Quote:
Originally Posted by ilikexd View Post

I do not think there should be an arbitrary limit of five at 40+ points. I can more sympathize with the limit of five back when batches would become full and close, but now that the batch is permanently open, I don't. If the simfile artist has truly proven him or herself by acquiring so many points, there should be even further cap tiers, as long as the points are a reliable measure.
I am up for considering a limit of 6 per two month period as I was considering doing it in the past; there should be more incentive for simfile artists to consistently produce high quality content. However, when considering the number of simfile artists that even have the 5 file cap, it seems a bit trivial given that only two people will meet the 6 file cap I have in mind.

Quote:
Originally Posted by ilikexd View Post
- That brings me to another point: I don't think they are that reliable. While having a lot of SAP is indeed a measure of simfiling capability, it becomes pretty inaccurate for a few reasons. If one in every three simfiles a simfile artist submits is accepted, he or she will be getting SAP over time, and his or her submission cap will raise indefinitely. But the intended purpose of the SAP system assuring quality control is not fulfilled; the opposite happens (this is an extreme theoretical example to demonstrate a point and doesn't happen to this extent in practice). Instead of SAP being granted for every file acceptance, even when those acceptances are accompanied by the same number or more rejections in a batch, SAP should be granted based on overall performance inside a single batch. One acceptance accompanied by more than one rejection is not a positive performance and should not yield +SAP.

I do not agree with this in practice, especially given the general volatility of the judgment team (this is something that I want to alleviate but cannot do just yet). If a person sends a less detailed, easier file that gets rejected and sends an extremely polished file that judges all like, it would make sense to put more weight on the acceptance of that better file; the SAP system does this. Additionally, a system where we judge user performance in their acceptance in a given batch has a massive potential to be ineffective given the fact that people can play it safe and continuously submit easier files, which arguably require less detail. Coincidentally enough, this conflicts with a general opinion that you had about people raising their cap with easy files -- if the system becomes performance based and will require consistency, people are more likely to submit files that are easier and require generally less detail. Alternatively, I can see users opting out of submitting as actively as they normally do simply because they are concerned about losing privileges despite an ability to get files accepted into the game. Do you think that this change in SAP to allow certain "privileged" stepartists while accommodating for everyone else is a good idea if it poses the risk of saturating the game with lowel level files? I don't think so.


Quote:
There is also no real way to lose SAP. Technically SAP is lost when receiving 1.5/10 or lower average on a simfile. But as far as I know this has never happened to any simfile artist who has any charts in game, so this measure is entirely useless. But it seems that based on its existence, somewhere along the line there was some intention to implement a system that punishes negative performance. Now to be sincere the actual prospect of having SAP reduced from a negative performance is pretty harsh, but as I wrote earlier I think SAP should be granted based on the overall/net performance in a batch, so that would mean considering both positive and negative performance within the same batch, which would not be a punishment/deduction in itself but would be so indirectly and would be an even better motivator for sending one's best simfiles and only one's best simfiles, and limit any frivolous submissions.
Adding deductions to the SAP system further beyond simple violations of the rules is incredibly unfair for those who are still trying to take the time to get their files through the system and get accepted. Against, this also correlates with the issue of judgment volatility -- just last batch, we had a file get two 10/10 [++] ratings and that same file also received a 6 [+?]; this isn't to say that any judgments are incorrect, but it speaks volumes about the fact that we can't rely on judgment scores even more heavily by basing increases in submission privileges off of multiple files. Additionally, while I am aware of certain people who are consistently able to produce good files, I am also aware of the fact that loosening/removing the cap will basically allow these users to saturate the game with their files, and I'm not a fan of that.
TC_Halogen is offline   Reply With Quote
Old 08-17-2014, 10:01 AM   #4
Wayward Vagabond
Confirmed Heartbreaker
Retired StaffFFR Simfile AuthorFFR Veteran
 
Wayward Vagabond's Avatar
 
Join Date: Jul 2012
Age: 35
Posts: 5,856
Default Re: Rework the SAP system

Whoever wants my sap points can buy them off me 10k credits per point
__________________
Wayward Vagabond is offline   Reply With Quote
Old 08-17-2014, 12:25 PM   #5
Wayward Vagabond
Confirmed Heartbreaker
Retired StaffFFR Simfile AuthorFFR Veteran
 
Wayward Vagabond's Avatar
 
Join Date: Jul 2012
Age: 35
Posts: 5,856
Default Re: Rework the SAP system

Sold
__________________
Wayward Vagabond is offline   Reply With Quote
Old 08-17-2014, 03:37 PM   #6
ilikexd
Retired StaffFFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
Join Date: Apr 2006
Posts: 3,205
Default Re: Rework the SAP system

Quote:
Originally Posted by TC_Halogen View Post
I do not agree with this in practice, especially given the general volatility of the judgment team (this is something that I want to alleviate but cannot do just yet)
I had thought this would come up, and I understand what you mean, but I don't really agree. The volatility already works in the opposite way by unjuistifiably granting +.

Quote:
Originally Posted by TC_Halogen View Post
If a person sends a less detailed, easier file that gets rejected and sends an extremely polished file that judges all like, it would make sense to put more weight on the acceptance of that better file; the SAP system does this.
If in a batch one file is rejected, and another file is accepted, and that acceptance is very strong, SAP can still be granted to the simfile artist in that batch. Just as the current SAP system grants +1-5 based on the strength of an acceptance, you can consider a rejection some negative points. Or you can consider light/minor rejections no negative points, which is often the case with easier files.

Quote:
Originally Posted by TC_Halogen View Post
Additionally, a system where we judge user performance in their acceptance in a given batch has a massive potential to be ineffective given the fact that people can play it safe and continuously submit easier files, which arguably require less detail.
Barely accepted files don't need to grant +SAP, then. In the current system any acceptance at all grants +1 SAP, and I don't argee with that for the reason you just mentioned.

If there is already a way to "game the system" in that respect, it's a separate issue.

Quote:
Originally Posted by TC_Halogen View Post
Coincidentally enough, this conflicts with a general opinion that you had about people raising their cap with easy files -- if the system becomes performance based and will require consistency, people are more likely to submit files that are easier and require generally less detail.
I think my proposition is being interpreted as judging based solely off a net acceptance/rejection count. I didn't mean it that way; I still think acceptances and rejections can be relatively worth more than each other: very strongly accepted files are still worth more than 1 SAP, barely accepted files can be worth few to none depending on an arbitrary cutoff, and barely rejected files don't need to be penalized at all.

Quote:
Originally Posted by TC_Halogen View Post
Alternatively, I can see users opting out of submitting as actively as they normally do simply because they are concerned about losing privileges despite an ability to get files accepted into the game. Do you think that this change in SAP to allow certain "privileged" stepartists while accommodating for everyone else is a good idea if it poses the risk of saturating the game with lowel level files? I don't think so.
Again, I think my proposition is being misinterpreted. I haven't called for any system by which simfile artists lose their SAP. What I'm saying is while the system is granting +SAP to positive performance it's neglecting all negative performance while doing so. Even in the case where a simfile artist has only rejections in a certain batch, I do not think the simfile artist should lose any SAP.

Quote:
Originally Posted by TC_Halogen View Post
Adding deductions to the SAP system further beyond simple violations of the rules is incredibly unfair for those who are still trying to take the time to get their files through the system and get accepted. Against, this also correlates with the issue of judgment volatility -- just last batch, we had a file get two 10/10 [++] ratings and that same file also received a 6 [+?]; this isn't to say that any judgments are incorrect, but it speaks volumes about the fact that we can't rely on judgment scores even more heavily by basing increases in submission privileges off of multiple files. Additionally, while I am aware of certain people who are consistently able to produce good files, I am also aware of the fact that loosening/removing the cap will basically allow these users to saturate the game with their files, and I'm not a fan of that.
I haven't proposed any deductions to SAP.
ilikexd is offline   Reply With Quote
Old 08-17-2014, 03:46 PM   #7
Deidara837
For a New Beginning
FFR Simfile AuthorFFR Veteran
 
Deidara837's Avatar
 
Join Date: Jun 2008
Location: New York, United States
Age: 28
Posts: 1,752
Default Re: Rework the SAP system

I have a couple questions as to the SAP system and timings of batches and the queue.

Say someone gets an average of 8.0 on a file (worth +2 SAP), and then that file is accepted into the queue. Does that person's SAP go up by 10 or 12? I'd assume it's 10, but I'm very new to the Queue/Batch system and the whole stepping thing in general.

Also, the main thread shows that the queue contains songs from up to the Mar/Apr 2014 Batch. The May/June 2014 Batch is closed, and probably doing judging/review atm. If a file in there would increase someone's SAP threshold to add a song submission for the next batch, would this kind of skip over the July/Aug 2014 Batch? The period is >3/4ths over, so by the time the additional submission is allowed the batch might be closed. Could it just be added to the July/Aug batch later or would it have to be put off until Sept/Oct?

Sorry if that was confusing, was a lil hard to word. Also sorry if this isn't a proper place to post this
__________________

_9th OT: D2 - 13th Place
10th OT: D5 - 45th Place
11th OT: D6 - 12th Place

EMG Rivalry | EMG Spreadsheet
Deidara837 is offline   Reply With Quote
Old 08-17-2014, 03:51 PM   #8
ilikexd
Retired StaffFFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
Join Date: Apr 2006
Posts: 3,205
Default Re: Rework the SAP system

Quote:
Originally Posted by Deidara837 View Post
I have a couple questions as to the SAP system and timings of batches and the queue.

Say someone gets an average of 8.0 on a file (worth +2 SAP), and then that file is accepted into the queue. Does that person's SAP go up by 10 or 12? I'd assume it's 10, but I'm very new to the Queue/Batch system and the whole stepping thing in general.

Also, the main thread shows that the queue contains songs from up to the Mar/Apr 2014 Batch. The May/June 2014 Batch is closed, and probably doing judging/review atm. If a file in there would increase someone's SAP threshold to add a song submission for the next batch, would this kind of skip over the July/Aug 2014 Batch? The period is >3/4ths over, so by the time the additional submission is allowed the batch might be closed. Could it just be added to the July/Aug batch later or would it have to be put off until Sept/Oct?

Sorry if that was confusing, was a lil hard to word. Also sorry if this isn't a proper place to post this
You get a bonus 10 points the very first time you have a simfile accepted, which boosts you into the second SAP tier. It only happens once. You'd also get whatever +SAP you get normally from the simfile, so 12 in that case.

@the second question: yes, if I understand what you asked correctly. SAP is granted as soon as batch notes are completed. So if you already submitted your cap of x for a period, then your cap raises from x to x+1 in the midddle of a period, you can still add 1 more before the end of that period.

Last edited by ilikexd; 08-17-2014 at 03:53 PM..
ilikexd is offline   Reply With Quote
Old 08-17-2014, 04:08 PM   #9
ilikexd
Retired StaffFFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
Join Date: Apr 2006
Posts: 3,205
Default Re: Rework the SAP system

Also, in regards to the max cap of 5 or 6:

I'd like to see tiers of 6 and 7 to be honest, not just 6, but I understand that we may just have differing opinions. I think capping at 5 after 40 points is too low given the amount of simfile artists who have doubled that amount, and adding 6 at 80-100 points would be a nice change.
ilikexd is offline   Reply With Quote
Old 08-17-2014, 04:12 PM   #10
TC_Halogen
Rhythm game specialist.
Retired StaffFFR Simfile AuthorFFR Music ProducerD8 Godly KeysmasherFFR Veteran
 
TC_Halogen's Avatar
 
Join Date: Feb 2008
Location: Bel Air, Maryland
Age: 32
Posts: 19,376
Send a message via AIM to TC_Halogen Send a message via Skype™ to TC_Halogen
Default Re: Rework the SAP system

Quote:
Originally Posted by ilikexd View Post
I had thought this would come up, and I understand what you mean, but I don't really agree. The volatility already works in the opposite way by unjuistifiably granting +.
Volatility is volatility -- you're mentioning that it works and I'm just mentioning it as a whole. In this case, it is significantly more important to mention volatility when lower ratings are the problem because they are likely the ones to inhibit a file's acceptance. A situation where volatility produces higher rankings as opposed to the more common lower one are not nearly as important because in most cases, those files do not get accepted. Improper ratings are the reason why something like this isn't easily feasible.

Quote:
Originally Posted by ilikexd View Post
If in a batch one file is rejected, and another file is accepted, and that acceptance is very strong, SAP can still be granted to the simfile artist in that batch. Just as the current SAP system grants +1-5 based on the strength of an acceptance, you can consider a rejection some negative points. Or you can consider light/minor rejections no negative points, which is often the case with easier files.
Getting a +4 rating is incredibly rare -- only one file has done it so far since the SAP system has existed, and +5 is likely not possible given deviations. If a person makes it past the cutoff of judgment, they deserve to gain points. I won't delve on this because this is an impasse waiting to happen -- these are just differences in how we feel the system should work and there's no purpose in elaborating.

Quote:
Originally Posted by ilikexd View Post
Barely accepted files don't need to grant +SAP, then. In the current system any acceptance at all grants +1 SAP, and I don't argee with that for the reason you just mentioned.
This is a much better point than what you said earlier, and I'm glad you said this -- however, I do have a response for it:

If a person becomes a major contributor to the site via this system and gets a large amount of files in game that are "barely accepted" due to general disagreement with judges (which by the way, we see this happen quite frequently), are they still going to have minimal SAP despite having a large number of files? I feel like this is a massive flaw in a new system like this, whereas the current system simply recognizes users for making a cutoff -- most users generally like this.

Let's think about the future a bit more too, because this is inevitably not only going to affect newer simfile artists with 2-file caps, but also the overall flow of files received:

Some of our currently rising simfile artists had to dip their feet in the water and receive a few lower ratings before starting to find their footing and creating their own style and I feel like the current system gives them that opportunity. A system that hands out lower SAP/more finely hands out SAP will inevitably constrain newer simfile artists, and I don't like that at all.

Quote:
If there is already a way to "game the system" in that respect, it's a separate issue.
b.) "already?" The situation in question is the new proposed system, not the current one. This admittedly is a problem no matter what gets done; if the SAP system stays the way it is, a user can increase their privilege through easier files -- if the SAP system changes, a user can generally increase their net gain by playing safe and just making solid easier files, which causes another saturation issue (granted, we could use a LOT more easier files, haha). This isn't to say that it won't happen, but most users who get their cap to three files start using that extra slot to try and experiment a bit more out of their comfort zone (a good example would be Xelnya/Middie sending in My Fxxkin Desire For You).

Quote:
I think my proposition is being interpreted as judging based solely off a net acceptance/rejection count. I didn't mean it that way; I still think acceptances and rejections can be relatively worth more than each other: very strongly accepted files are still worth more than 1 SAP, barely accepted files can be worth few to none depending on an arbitrary cutoff, and barely rejected files don't need to be penalized at all.


Again, I think my proposition is being misinterpreted. I haven't called for any system by which simfile artists lose their SAP. What I'm saying is while the system is granting +SAP to positive performance it's neglecting all negative performance while doing so. Even in the case where a simfile artist has only rejections in a certain batch, I do not think the simfile artist should lose any SAP.

I haven't proposed any deductions to SAP.
There was a general implication in your post here:
Quote:
But it seems that based on its existence, somewhere along the line there was some intention to implement a system that punishes negative performance. Now to be sincere the actual prospect of having SAP reduced from a negative performance is pretty harsh, but as I wrote earlier I think SAP should be granted based on the overall/net performance in a batch, so that would mean considering both positive and negative performance within the same batch, which would not be a punishment/deduction in itself but would be so indirectly and would be an even better motivator for sending one's best simfiles and only one's best simfiles, and limit any frivolous submissions.
Another inherent problem is that this is being mentioned by you (and this is not meant for you to take offense): you have the highest success rate out of any simfile artist on the site; not one single file you have sent in was fully rejected and the list of files you have is building quite quickly. You knew what you were getting into early on, but:

Quote:
but would be so indirectly and would be an even better motivator for sending one's best simfiles and only one's best simfiles, and limit any frivolous submissions.
If we're going to promote net performance, we're going to lead to smaller gains and less people moving up the SAP ladder. This will snowball to less files sent in because people want to optimize their net performance; some people will be afraid to submit and eventually, we might expect the total number of files received (and accepted) to reduce a bit. The point about saturation from more experienced stepartists will become more and more prevalent as time goes on -- high quality simfile artists will continue to gain from a system like this, while those who are barely getting by have very little incentive to keep submitting. If they feel like it takes too much time to wait, what's the point?
TC_Halogen is offline   Reply With Quote
Old 08-17-2014, 04:16 PM   #11
TC_Halogen
Rhythm game specialist.
Retired StaffFFR Simfile AuthorFFR Music ProducerD8 Godly KeysmasherFFR Veteran
 
TC_Halogen's Avatar
 
Join Date: Feb 2008
Location: Bel Air, Maryland
Age: 32
Posts: 19,376
Send a message via AIM to TC_Halogen Send a message via Skype™ to TC_Halogen
Default Re: Rework the SAP system

Quote:
Originally Posted by ilikexd View Post
Also, in regards to the max cap of 5 or 6:

I'd like to see tiers of 6 and 7 to be honest, not just 6, but I understand that we may just have differing opinions. I think capping at 5 after 40 points is too low given the amount of simfile artists who have doubled that amount, and adding 6 at 80-100 points would be a nice change.
Given that the system accommodates for files sent bi-monthly, I am not opposed to a tier of 6 and would be willing to put that in effect with a SAP total of 70, but I'm not doing something like that without talking to JX/bmah; while I might be the one appearing to do things at face value, there are two other batch heads that need to be approving of this, along with anything else that is done.
TC_Halogen is offline   Reply With Quote
Old 08-17-2014, 04:38 PM   #12
ilikexd
Retired StaffFFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
Join Date: Apr 2006
Posts: 3,205
Default Re: Rework the SAP system

Admittedly the main purpose when I was planning on posting this was additional tiers, not the point system itself, so I hadn't considered a lot of those implications from a net-based SAP system, and can see now why they would have negative consequences such as saturation and reluctance to submit.
ilikexd is offline   Reply With Quote
Old 08-18-2014, 12:10 AM   #13
AutotelicBrown
Under the scarlet moon
FFR Simfile AuthorD7 Elite KeysmasherFFR Veteran
 
AutotelicBrown's Avatar
 
Join Date: Jan 2014
Age: 31
Posts: 921
Default Re: Rework the SAP system

The main problems I see with the current system is that SAP gains after you achieve 40 (or any arbitrary top tier) are essentially useless as SAP deductions are unlikely to happen, and that prolific steppers aren't able to push a few more files into the game if the work is truly top-notch.

What about introducing a few more tiers with an added change that, if you want to make use of them (i.e. submitting more than 5 files), you have to go through a progressively harsher SAP bonus/deduction table up to a soft limit where a 10/10 file would net 0 points and anything lower gives a proportional deduction to SAP.

Just as an example, a table of SAP changes according to the number of files being submitted for the batch and obtained average score (PS: I know those values are bad):

--------| 1-5 files (current values) | 6 files | 7 files | 8 files
9.5-10. | +5 | +4 | +2 | +1
9.0-9.5 | +4 | +3 | +1 | 0
8.3-9.0 | +3 | +2 | 0 | -1
7.5-8.3 | +2 | +1 | -1 | -4
6.7-7.5 | +1 | 0 | -3 | -7
1.5-6.7 | 0 | -1 | -5 | -10
1.0-1.5 | -1 | -2 | -7 | -13
0.0-1.0 | -2 | -3 | -9 | -16

The benefits from that would be:
-Doesn't change what already works
-SAP gains are always useful as you can 'spend' them sending more files
-Actual incentive to make top-notch files
-Don't hinder experimentation as in the worst case scenario you are at the base 5 files limit

It softens the saturation problem as the stepper is more likely to spend SAP submitting a higher number of files. In that case the stepper wouldn't be able to sustain a high number of submissions per batch.
Unless the stepper can consistently push godlike scores on a reasonable amount of files under a volatile judgement system, in which case I don't see why the game shouldn't get a few more top-quality files than usual.

The idea is really raw, I have no idea what would be good values and I don't have a holistic view of the system, but I guess it doesn't hurt to share it.
AutotelicBrown is offline   Reply With Quote
Old 08-18-2014, 09:56 AM   #14
jimerax
Retired StaffFFR Simfile AuthorFFR Music ProducerFFR Veteran
 
jimerax's Avatar
 
Join Date: Nov 2003
Location: Tokyo
Posts: 8,185
Send a message via AIM to jimerax Send a message via MSN to jimerax Send a message via Skype™ to jimerax
Default Re: Rework the SAP system

Responding with this issue.

Quote:
Originally Posted by ilikexd View Post
I do not think there should be an arbitrary limit of five at 40+ points. I can more sympathize with the limit of five back when batches would become full and close, but now that the batch is permanently open, I don't. If the simfile artist has truly proven him or herself by acquiring so many points, there should be even further cap tiers, as long as the points are a reliable measure.
Submission cap/SAP requirement can always be adjusted with judge's status, so it will be fine depends on judgement man-power.
Once our judgement team catches up with the realtime batch then it can definitely be changed.
Again, depends on judge's conditions.


Quote:
Originally Posted by ilikexd View Post
That brings me to another point: I don't think they are that reliable. While having a lot of SAP is indeed a measure of simfiling capability, it becomes pretty inaccurate for a few reasons. If one in every three simfiles a simfile artist submits is accepted, he or she will be getting SAP over time, and his or her submission cap will raise indefinitely. But the intended purpose of the
SAP system assuring quality control is not fulfilled; the opposite happens (this is an extreme theoretical example to demonstrate a point and doesn't happen to this extent in practice). Instead of SAP being granted for every file acceptance, even when those acceptances are accompanied by the same number or more rejections in a batch, SAP should be granted based on overall performance inside a single batch. One acceptance accompanied by more than one rejection is not a positive performance and should not yield +SAP.
Yeah rejected files / average rating / submission number can be considered.. just need appropriate calculation system.

A quite complicated formula but for example,
([# of rejected files] - [average rating - 6.5]) * [# of submissions - 2]^0.5 as SAP penalty
(if one of parameters gets negative then the penalty counted as 0, fractions below decimal point are ignored).

submission/acceptance/rejection count of collab file is 0.5, as usual.


Quote:
Originally Posted by ilikexd View Post
There is also no real way to lose SAP. Technically SAP is lost when receiving 1.5/10 or lower average on a simfile. But as far as I know this has never happened to any simfile artist who has any charts in game, so this measure is entirely useless. But it seems that based on its existence, somewhere along the line there was some intention to implement a system that punishes negative performance. Now to be sincere the actual prospect of having SAP reduced from a negative performance is pretty harsh, but as I wrote earlier I think SAP should be granted based on the overall/net performance in a batch, so that would mean considering both positive and negative performance within the same batch, which would not be a punishment/deduction in itself but would be so indirectly and would be an even better motivator for sending one's best simfiles and only one's best
simfiles, and limit any frivolous submissions.
Penalties like that were set before (like 1/10 or something), but it was rarely applied for people who have many SAPs.

There can be adjustments for people who have high SAPs (just like FFR multiplayer level).

ex. to get 5/4/3/2 points
Cap=6 - 9.5/9.2/8.7/8.0
Cap=5 - 9.5/9.1/8.6/7.9
Cap=4 - 9.5/9.1/8.5/7.8
Cap=3 - 9.5/9.0/8.4/7.6
Cap=2 - 9.5/9.0/8.3/7.5

Of course acceptance line should be the same for all submitters, since it's no good for the ingame file quality.

and in collab files, we will use parameters from the stepper who has the highest SAPs/submission cap for SAP adjestment calculation written above.


Quote:
Originally Posted by AutotelicBrown View Post
--------| 1-5 files (current values) | 6 files | 7 files | 8 files
9.5-10. | +5 | +4 | +2 | +1
9.0-9.5 | +4 | +3 | +1 | 0
8.3-9.0 | +3 | +2 | 0 | -1
7.5-8.3 | +2 | +1 | -1 | -4
6.7-7.5 | +1 | 0 | -3 | -7
1.5-6.7 | 0 | -1 | -5 | -10
1.0-1.5 | -1 | -2 | -7 | -13
0.0-1.0 | -2 | -3 | -9 | -16

The benefits from that would be:
-Doesn't change what already works
-SAP gains are always useful as you can 'spend' them sending more files
-Actual incentive to make top-notch files
-Don't hinder experimentation as in the worst case scenario you are at the base 5 files limit
Yeah can add something like this penalty system, allowing +1-2 extra submittions (submitters will lose many SAPs if files don't meat quality requirement).


Also another problem, SAP for collab file issue.
We have 2 options, will listen to opinions here (not using decimal point).

1. no SAPs as before
2. half/one third SAPs for each collaborators (fractions below decimal point are ignored, ex: if the file submitted as collab by 2 steppers gets only 1 SAP, or file submitted as collab by 3 steppers gets only 2 SAPs then no point for each submitter).

Last edited by jimerax; 08-18-2014 at 10:04 AM..
jimerax is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -5. The time now is 03:50 AM.


Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright FlashFlashRevolution