08-17-2014, 05:54 AM | #1 |
Join Date: Apr 2006
Posts: 3,205
|
Rework the SAP system
The SAP system is a clear necessity in the FFR submission process to assure quality control. But I think there is a problem in how points are collected and in the submission cap tiers.
The idea behind submission caps, as I understand it, is as follows: Everyone should have the opportunity to submit simfiles to FFR, but the judgement process is a long and laborious one, so in order to save time and effort for simfile judges and batch organizers, simfile artists are restricted to a certain number of simfiles they can send per period. Naturally, this compels them to only send simfiles that they think are their best and have a chance at being accepted, thus raising the average quality of all submissions. When a simfile artist gets many simfiles accepted, his or her submission cap is raised based on the idea that the simfiles being produced are of a higher quality and less likely to be rejected. The problem is that the whole concept of a higher cap for the purpose of having a higher ratio of acceptance-worthy to rejection-prone simfiles in a batch isn't being executed optimally. I do not think there should be an arbitrary limit of five at 40+ points. I can more sympathize with the limit of five back when batches would become full and close, but now that the batch is permanently open, I don't. If the simfile artist has truly proven him or herself by acquiring so many points, there should be even further cap tiers, as long as the points are a reliable measure. That brings me to another point: I don't think they are that reliable. While having a lot of SAP is indeed a measure of simfiling capability, it becomes pretty inaccurate for a few reasons. If one in every three simfiles a simfile artist submits is accepted, he or she will be getting SAP over time, and his or her submission cap will raise indefinitely. But the intended purpose of the SAP system assuring quality control is not fulfilled; the opposite happens (this is an extreme theoretical example to demonstrate a point and doesn't happen to this extent in practice). Instead of SAP being granted for every file acceptance, even when those acceptances are accompanied by the same number or more rejections in a batch, SAP should be granted based on overall performance inside a single batch. One acceptance accompanied by more than one rejection is not a positive performance and should not yield +SAP. There is also no real way to lose SAP. Technically SAP is lost when receiving 1.5/10 or lower average on a simfile. But as far as I know this has never happened to any simfile artist who has any charts in game, so this measure is entirely useless. But it seems that based on its existence, somewhere along the line there was some intention to implement a system that punishes negative performance. Now to be sincere the actual prospect of having SAP reduced from a negative performance is pretty harsh, but as I wrote earlier I think SAP should be granted based on the overall/net performance in a batch, so that would mean considering both positive and negative performance within the same batch, which would not be a punishment/deduction in itself but would be so indirectly and would be an even better motivator for sending one's best simfiles and only one's best simfiles, and limit any frivolous submissions. Last edited by ilikexd; 08-17-2014 at 05:56 AM.. |
08-17-2014, 08:21 AM | #2 |
Simfile Judge
Join Date: May 2013
Posts: 476
|
Re: Rework the SAP system
If all quality step artists get like 10 SAP, how much room is there for new artists if all the good step artists inevitably raise the bar. You can't really expect the same results from a stepartist who has been stepping for 5 years, and a stepartist who maybe has been stepping for only 3 months. But what happens then? Do you accept all 10 files? This will inevitably make the judges more strict on accepting files making it even harder for the new guys.
|
08-17-2014, 09:08 AM | #3 | |||
Rhythm game specialist.
|
Re: Rework the SAP system
Quote:
Quote:
I do not agree with this in practice, especially given the general volatility of the judgment team (this is something that I want to alleviate but cannot do just yet). If a person sends a less detailed, easier file that gets rejected and sends an extremely polished file that judges all like, it would make sense to put more weight on the acceptance of that better file; the SAP system does this. Additionally, a system where we judge user performance in their acceptance in a given batch has a massive potential to be ineffective given the fact that people can play it safe and continuously submit easier files, which arguably require less detail. Coincidentally enough, this conflicts with a general opinion that you had about people raising their cap with easy files -- if the system becomes performance based and will require consistency, people are more likely to submit files that are easier and require generally less detail. Alternatively, I can see users opting out of submitting as actively as they normally do simply because they are concerned about losing privileges despite an ability to get files accepted into the game. Do you think that this change in SAP to allow certain "privileged" stepartists while accommodating for everyone else is a good idea if it poses the risk of saturating the game with lowel level files? I don't think so. Quote:
|
|||
08-17-2014, 10:01 AM | #4 |
Confirmed Heartbreaker
Join Date: Jul 2012
Age: 35
Posts: 5,859
|
Re: Rework the SAP system
Whoever wants my sap points can buy them off me 10k credits per point
__________________
|
08-17-2014, 12:25 PM | #5 |
Confirmed Heartbreaker
Join Date: Jul 2012
Age: 35
Posts: 5,859
|
Re: Rework the SAP system
Sold
__________________
|
08-17-2014, 03:37 PM | #6 | ||||||
Join Date: Apr 2006
Posts: 3,205
|
Re: Rework the SAP system
Quote:
Quote:
Quote:
If there is already a way to "game the system" in that respect, it's a separate issue. Quote:
Quote:
Quote:
|
||||||
08-17-2014, 03:46 PM | #7 |
For a New Beginning
Join Date: Jun 2008
Location: New York, United States
Age: 28
Posts: 1,752
|
Re: Rework the SAP system
I have a couple questions as to the SAP system and timings of batches and the queue.
Say someone gets an average of 8.0 on a file (worth +2 SAP), and then that file is accepted into the queue. Does that person's SAP go up by 10 or 12? I'd assume it's 10, but I'm very new to the Queue/Batch system and the whole stepping thing in general. Also, the main thread shows that the queue contains songs from up to the Mar/Apr 2014 Batch. The May/June 2014 Batch is closed, and probably doing judging/review atm. If a file in there would increase someone's SAP threshold to add a song submission for the next batch, would this kind of skip over the July/Aug 2014 Batch? The period is >3/4ths over, so by the time the additional submission is allowed the batch might be closed. Could it just be added to the July/Aug batch later or would it have to be put off until Sept/Oct? Sorry if that was confusing, was a lil hard to word. Also sorry if this isn't a proper place to post this |
08-17-2014, 03:51 PM | #8 | |
Join Date: Apr 2006
Posts: 3,205
|
Re: Rework the SAP system
Quote:
@the second question: yes, if I understand what you asked correctly. SAP is granted as soon as batch notes are completed. So if you already submitted your cap of x for a period, then your cap raises from x to x+1 in the midddle of a period, you can still add 1 more before the end of that period. Last edited by ilikexd; 08-17-2014 at 03:53 PM.. |
|
08-17-2014, 04:08 PM | #9 |
Join Date: Apr 2006
Posts: 3,205
|
Re: Rework the SAP system
Also, in regards to the max cap of 5 or 6:
I'd like to see tiers of 6 and 7 to be honest, not just 6, but I understand that we may just have differing opinions. I think capping at 5 after 40 points is too low given the amount of simfile artists who have doubled that amount, and adding 6 at 80-100 points would be a nice change. |
08-17-2014, 04:12 PM | #10 | |||||||
Rhythm game specialist.
|
Re: Rework the SAP system
Quote:
Quote:
Quote:
If a person becomes a major contributor to the site via this system and gets a large amount of files in game that are "barely accepted" due to general disagreement with judges (which by the way, we see this happen quite frequently), are they still going to have minimal SAP despite having a large number of files? I feel like this is a massive flaw in a new system like this, whereas the current system simply recognizes users for making a cutoff -- most users generally like this. Let's think about the future a bit more too, because this is inevitably not only going to affect newer simfile artists with 2-file caps, but also the overall flow of files received: Some of our currently rising simfile artists had to dip their feet in the water and receive a few lower ratings before starting to find their footing and creating their own style and I feel like the current system gives them that opportunity. A system that hands out lower SAP/more finely hands out SAP will inevitably constrain newer simfile artists, and I don't like that at all. Quote:
Quote:
Quote:
Quote:
|
|||||||
08-17-2014, 04:16 PM | #11 | |
Rhythm game specialist.
|
Re: Rework the SAP system
Quote:
|
|
08-17-2014, 04:38 PM | #12 |
Join Date: Apr 2006
Posts: 3,205
|
Re: Rework the SAP system
Admittedly the main purpose when I was planning on posting this was additional tiers, not the point system itself, so I hadn't considered a lot of those implications from a net-based SAP system, and can see now why they would have negative consequences such as saturation and reluctance to submit.
|
08-18-2014, 12:10 AM | #13 |
Under the scarlet moon
Join Date: Jan 2014
Age: 31
Posts: 921
|
Re: Rework the SAP system
The main problems I see with the current system is that SAP gains after you achieve 40 (or any arbitrary top tier) are essentially useless as SAP deductions are unlikely to happen, and that prolific steppers aren't able to push a few more files into the game if the work is truly top-notch.
What about introducing a few more tiers with an added change that, if you want to make use of them (i.e. submitting more than 5 files), you have to go through a progressively harsher SAP bonus/deduction table up to a soft limit where a 10/10 file would net 0 points and anything lower gives a proportional deduction to SAP. Just as an example, a table of SAP changes according to the number of files being submitted for the batch and obtained average score (PS: I know those values are bad): --------| 1-5 files (current values) | 6 files | 7 files | 8 files 9.5-10. | +5 | +4 | +2 | +1 9.0-9.5 | +4 | +3 | +1 | 0 8.3-9.0 | +3 | +2 | 0 | -1 7.5-8.3 | +2 | +1 | -1 | -4 6.7-7.5 | +1 | 0 | -3 | -7 1.5-6.7 | 0 | -1 | -5 | -10 1.0-1.5 | -1 | -2 | -7 | -13 0.0-1.0 | -2 | -3 | -9 | -16 The benefits from that would be: -Doesn't change what already works -SAP gains are always useful as you can 'spend' them sending more files -Actual incentive to make top-notch files -Don't hinder experimentation as in the worst case scenario you are at the base 5 files limit It softens the saturation problem as the stepper is more likely to spend SAP submitting a higher number of files. In that case the stepper wouldn't be able to sustain a high number of submissions per batch. Unless the stepper can consistently push godlike scores on a reasonable amount of files under a volatile judgement system, in which case I don't see why the game shouldn't get a few more top-quality files than usual. The idea is really raw, I have no idea what would be good values and I don't have a holistic view of the system, but I guess it doesn't hurt to share it. |
08-18-2014, 09:56 AM | #14 | ||||
★★★★★
|
Re: Rework the SAP system
Responding with this issue.
Quote:
Once our judgement team catches up with the realtime batch then it can definitely be changed. Again, depends on judge's conditions. Quote:
A quite complicated formula but for example, ([# of rejected files] - [average rating - 6.5]) * [# of submissions - 2]^0.5 as SAP penalty (if one of parameters gets negative then the penalty counted as 0, fractions below decimal point are ignored). submission/acceptance/rejection count of collab file is 0.5, as usual. Quote:
There can be adjustments for people who have high SAPs (just like FFR multiplayer level). ex. to get 5/4/3/2 points Cap=6 - 9.5/9.2/8.7/8.0 Cap=5 - 9.5/9.1/8.6/7.9 Cap=4 - 9.5/9.1/8.5/7.8 Cap=3 - 9.5/9.0/8.4/7.6 Cap=2 - 9.5/9.0/8.3/7.5 Of course acceptance line should be the same for all submitters, since it's no good for the ingame file quality. and in collab files, we will use parameters from the stepper who has the highest SAPs/submission cap for SAP adjestment calculation written above. Quote:
Also another problem, SAP for collab file issue. We have 2 options, will listen to opinions here (not using decimal point). 1. no SAPs as before 2. half/one third SAPs for each collaborators (fractions below decimal point are ignored, ex: if the file submitted as collab by 2 steppers gets only 1 SAP, or file submitted as collab by 3 steppers gets only 2 SAPs then no point for each submitter). Last edited by jimerax; 08-18-2014 at 10:04 AM.. |
||||
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|