[SM5] xwidghet's Custom Build

PolygonEater · 08-26-2016, 02:54 AM

Have you ever started up stepmania 5, and been confused as to why the frame rate is abysmal? Does using rates make you die inside when the music pans to the right, or when stepmania crashes beyond 1.6? Well have I got the solution for you...

Custom SM5 Build: https://github.com/xwidghet/stepmania/releases/latest
Minimum Requirements: Windows 7 or newer Operating System.

Performance example

Cataclysm- · 08-28-2016, 01:09 PM

**tosh** · 08-28-2016, 07:14 PM

I'm getting about 20% worse performance compared to the official 5.0.11 build (120 fps compared to 150 fps) and 50% worse performance than my 5.0.7 build (120 fps compared to 240 fps). Also, D3D on your custom build capped the frame rate for me at 60 fps even though I had vsync disabled and Aero turned on.

Is the exact .exe in the github link required for your custom build to work correctly? I tried using it and my computer kept telling me I didn't have permission to use it (even when I tried to run it as administrator).

Edit: Just noticed the 5.0.12 in the requirements list. Will try it with that and then see how it works.
Edit 2: So the issue I was having earlier is that my antivirus program thought that the custom build and base 5.0.12 executables were viruses. We're good now. Getting about 320 fps with D3D now with the default theme. About 500 fps if using a decent theme.

MinaciousGrace · 08-31-2016, 07:20 AM

Excellent magical bumperino. With xwidgets help and some startling revelations about how stupid I am and how stupid that makes certain other people by consequence, the build is updated more or less to the point of when the screenshot in the other thread was taken.

Feel free to read through the commits or just enjoy your ~1000 extra frames per second, up to you.

PolygonEater · 09-11-2016, 04:23 AM

As of this release, I've enabled Issues on github. If you run into problems which don't occur on Stepmania 5.0.12 please submit them there with a list of steps on how to reproduce them. If the game crashes please include your log file. We can't really fix bugs if we don't run into them ourselves and you don't tell us...unless we happen to discover them while reading the code. So please, actually submit them so we know. c: Though if you're having issues because you dirty installed Stepmania 5.0.12 over another version of stepmania I will find you...and I will...make you play noodle files at 2fps on a membrane keyboard with justice judgments. >:(

Locations of log files:

If you are using a portable.ini, then inside your Stepmania 5 folder there will be a Logs folder.

If you are not using a portable.ini, then it will be inside C:/Users/YourUsernameHere/AppData/Roaming/Stepmania 5/Logs

https://github.com/xwidghet/stepmania/issues

TyroneSama · 10-7-2016, 04:59 AM

Are there any plans to add the loading improvements from 5.1 into this build? I love the performance in-game, but it takes about 3 minutes to load all my packs, compared to about 15 seconds on 5.1.

MinaciousGrace · 10-7-2016, 09:54 PM

afaik 5.1 doesn't specifically have any optimization for song load

There are changes that slow down load time in the custom build so that chart key generation is frontloaded rather than done during song select, however these should be negligible. My load times are ~3 seconds for 5.1 versus ~7 for 5.0.12 custom.

If your differential is that great you probably forgot to re-enable fast load or fast load for additional songs.

that being said the code that was added isn't efficient by any means and I do plan to clean it up as well as make optimizations to song load times in other capacities

TyroneSama · 10-8-2016, 09:17 AM

Quote:

Originally Posted by MinaciousGrace

afaik 5.1 doesn't specifically have any optimization for song load

I thought for sure I remembered someone talking about it: definitely noticed a huge difference even pre-xwidghet build. I'm probably wrong, though, haven't been keeping up with SM dev things lately.

Quote:

Originally Posted by MinaciousGrace

If your differential is that great you probably forgot to re-enable fast load or fast load for additional songs.

They're both on. I'll poke around and see if I can work out what's going wrong.

MinaciousGrace · 10-8-2016, 11:30 AM

Both xwidget and I skimmed the 5.1 changelogs when we were deciding which point to fork the build from and it became apparent pretty early on it wasn't worth touching anything in the 5.1 line so it's possible there are some and I just missed them.

The only way that songload should take 3 minutes is if all or the majority of files aren't being cached on load. I noticed this was true for a significant portion of older files such as files from the astronomic metal pack/hsmp pack era. So if you have a large number of older files or files that inherit whatever condition prevents a cache file from being built it's possible that you have disproportionate slowdown for song load compared to other people. It's also possible that you have some sort of permissions issue that prevents the cache from being built (or read?) in entirety, which would effectively nullify fastload.

Longer files that contain lots of bpm changes (ddream syndrome lookin at u) also tend to take exponentially longer to load compared to 60 second 1 bpm and done files so if you have a lot of those it probably doesn't help, but I doubt it would be the main issue. All of my files are bpm corrected so I don't feel the effects of that nearly as much as someone who doesn't would.

I'll poke around at song load tomorrow; it would be helpful to have more info on what files you're loading so I can have a better idea of what to prioritize. Also whether or not all the files take similarly long to load or if a bunch blaze by until you hit a very slow load repeatedly.

Really the entire caching system needs to be rewritten and the .sm format probably has the lowest information:size ratio of any data format in existence but uhh, one bandaid at a time I guess.

TyroneSama · 10-16-2016, 12:55 PM

Honestly, the weirdest thing about the process is how StepMania decides to randomly throw out whatever index it's built from the song folders. The first time I run it in any given 12-hour-ish period, even with no changes to the song folder, it gives me The Long Load Of Death -- where, yes, it seems like it's stalling for multiple seconds on certain songs, always the same ones. I'll see if I can set up a screen recording or something to catch which ones it is.

After that's done, subsequent launches are near-instant...until StepMania decides that it's tired of being fast and goes through The Long Load again. Really don't know what's going on there.

MinaciousGrace · 10-17-2016, 12:15 AM

the game for whatever reason wasn't writing entries into an index file as it cached each song (which means interrupting the song caching process at 99% would result in starting from scratch upon restarting, also technically it did, but only for banners and not songs?), xwidget fixed this along with making other substantial improvements to songload

first time song load times should be cut by about 40% (numbers kinda wobble depending on whether or not you're running off a hard disk or ssd) and subsequent loads should also be slightly faster

but that issue is separate from yours i guess; the cache files are located in save/cache, not song, so that's what you should be monitoring. The only thing I can think of is that your cache somehow gets deleted upon reboot or something, or permissions get megafukt and the game can't read the files???

But that wouldn't explain why it only happens with the custom build and not the base. None of the io functions to the file system were modified in any way apart from what was being written in them. If it's irritating enough a possible solution would be to keep a backup of your cache folder somewhere else on your hd and paste it back any time you're having the long load issue. That's as much as I can guess at without being able to reproduce the issue myself :/

TyroneSama · 10-17-2016, 07:32 PM

I'm looking back through earlier posts and I have no idea why I implied that the issue was specific to the custom build. It happens on vanilla 5.0.12 for me as well. Sorry for being a potato.

In any case, the behavior I'm getting seems consistent with cache problems, but I can't figure out what causes the cache to get thrown out -- rebooting isn't what's triggering it, and none of the files in Roaming/Stepmania 5/Cache/Songs seem to be disappearing or getting overwritten (according to their last-modified date). Moved everything to my new SSD, so that hopefully rules out a failing or misbehaving drive, and I even tried deleting the cache manually so that SM could rebuild it; lo and behold, death load a few hours later.

I didn't really mean for this to turn into an extended General Stepmania Tech Support exchange, so sorry for shitting up the thread. If there really are no loading changes in 5.1, I'll keep poking around for an explanation.

PolygonEater · 10-17-2016, 11:45 PM

Quote:

Originally Posted by TyroneSama

If there really are no loading changes in 5.1, I'll keep poking around for an explanation.

Stepmania loading, at least in the custom build with the current optimizations I've added, is limited by it trying to get values to write or read for the cache. I have now merged changes from 5.1, but since they are eclipsed by the aforementioned problem they don't really make any difference. There should also only be a small subset of your collection which trigger re-caching. That being said please make sure both of the following values are set in your preferences ini unless you load songs directly from the Stepmania folder you installed to.

FastLoad=1
FastLoadAdditionalSongs=1

08-26-2016, 02:54 AM	#1
PolygonEater FFR Player Join Date: Jul 2011 Posts: 67	[SM5] xwidghet's Custom Build Have you ever started up stepmania 5, and been confused as to why the frame rate is abysmal? Does using rates make you die inside when the music pans to the right, or when stepmania crashes beyond 1.6? Well have I got the solution for you... Custom SM5 Build: https://github.com/xwidghet/stepmania/releases/latest Minimum Requirements: Windows 7 or newer Operating System. Performance example Last edited by PolygonEater; 12-22-2016 at 07:06 AM..

08-28-2016, 01:09 PM	#2
Cataclysm- FFR Player Join Date: Dec 2015 Age: 26 Posts: 174	Re: [SM5] xwidghet's Custom Build

08-28-2016, 07:14 PM	#3
tosh O Derby, Where Art Thou? Join Date: Jul 2003 Location: Virginia Age: 32 Posts: 1,011	Re: [SM5] xwidghet's Custom Build I'm getting about 20% worse performance compared to the official 5.0.11 build (120 fps compared to 150 fps) and 50% worse performance than my 5.0.7 build (120 fps compared to 240 fps). Also, D3D on your custom build capped the frame rate for me at 60 fps even though I had vsync disabled and Aero turned on. Is the exact .exe in the github link required for your custom build to work correctly? I tried using it and my computer kept telling me I didn't have permission to use it (even when I tried to run it as administrator). Edit: Just noticed the 5.0.12 in the requirements list. Will try it with that and then see how it works. Edit 2: So the issue I was having earlier is that my antivirus program thought that the custom build and base 5.0.12 executables were viruses. We're good now. Getting about 320 fps with D3D now with the default theme. About 500 fps if using a decent theme. Last edited by tosh; 08-28-2016 at 09:46 PM..

08-31-2016, 07:20 AM	#4
MinaciousGrace FFR Player Join Date: Dec 2007 Location: nima Posts: 4,278	Re: [SM5] xwidghet's Custom Build Excellent magical bumperino. With xwidgets help and some startling revelations about how stupid I am and how stupid that makes certain other people by consequence, the build is updated more or less to the point of when the screenshot in the other thread was taken. Feel free to read through the commits or just enjoy your ~1000 extra frames per second, up to you.

09-11-2016, 04:23 AM	#5
PolygonEater FFR Player Join Date: Jul 2011 Posts: 67	Re: [SM5] xwidghet's Custom Build As of this release, I've enabled Issues on github. If you run into problems which don't occur on Stepmania 5.0.12 please submit them there with a list of steps on how to reproduce them. If the game crashes please include your log file. We can't really fix bugs if we don't run into them ourselves and you don't tell us...unless we happen to discover them while reading the code. So please, actually submit them so we know. c: Though if you're having issues because you dirty installed Stepmania 5.0.12 over another version of stepmania I will find you...and I will...make you play noodle files at 2fps on a membrane keyboard with justice judgments. >:( Locations of log files: If you are using a portable.ini, then inside your Stepmania 5 folder there will be a Logs folder. If you are not using a portable.ini, then it will be inside C:/Users/YourUsernameHere/AppData/Roaming/Stepmania 5/Logs https://github.com/xwidghet/stepmania/issues Last edited by PolygonEater; 09-11-2016 at 04:27 AM..

10-7-2016, 04:59 AM	#6
TyroneSama FFR Player Join Date: Aug 2016 Posts: 8	Re: [SM5] xwidghet's Custom Build Are there any plans to add the loading improvements from 5.1 into this build? I love the performance in-game, but it takes about 3 minutes to load all my packs, compared to about 15 seconds on 5.1.

10-7-2016, 09:54 PM	#7
MinaciousGrace FFR Player Join Date: Dec 2007 Location: nima Posts: 4,278	Re: [SM5] xwidghet's Custom Build afaik 5.1 doesn't specifically have any optimization for song load There are changes that slow down load time in the custom build so that chart key generation is frontloaded rather than done during song select, however these should be negligible. My load times are ~3 seconds for 5.1 versus ~7 for 5.0.12 custom. If your differential is that great you probably forgot to re-enable fast load or fast load for additional songs. that being said the code that was added isn't efficient by any means and I do plan to clean it up as well as make optimizations to song load times in other capacities

10-8-2016, 11:30 AM	#9
MinaciousGrace FFR Player Join Date: Dec 2007 Location: nima Posts: 4,278	Re: [SM5] xwidghet's Custom Build Both xwidget and I skimmed the 5.1 changelogs when we were deciding which point to fork the build from and it became apparent pretty early on it wasn't worth touching anything in the 5.1 line so it's possible there are some and I just missed them. The only way that songload should take 3 minutes is if all or the majority of files aren't being cached on load. I noticed this was true for a significant portion of older files such as files from the astronomic metal pack/hsmp pack era. So if you have a large number of older files or files that inherit whatever condition prevents a cache file from being built it's possible that you have disproportionate slowdown for song load compared to other people. It's also possible that you have some sort of permissions issue that prevents the cache from being built (or read?) in entirety, which would effectively nullify fastload. Longer files that contain lots of bpm changes (ddream syndrome lookin at u) also tend to take exponentially longer to load compared to 60 second 1 bpm and done files so if you have a lot of those it probably doesn't help, but I doubt it would be the main issue. All of my files are bpm corrected so I don't feel the effects of that nearly as much as someone who doesn't would. I'll poke around at song load tomorrow; it would be helpful to have more info on what files you're loading so I can have a better idea of what to prioritize. Also whether or not all the files take similarly long to load or if a bunch blaze by until you hit a very slow load repeatedly. Really the entire caching system needs to be rewritten and the .sm format probably has the lowest information:size ratio of any data format in existence but uhh, one bandaid at a time I guess. Last edited by MinaciousGrace; 10-8-2016 at 11:32 AM..

10-16-2016, 12:55 PM	#10
TyroneSama FFR Player Join Date: Aug 2016 Posts: 8	Re: [SM5] xwidghet's Custom Build Honestly, the weirdest thing about the process is how StepMania decides to randomly throw out whatever index it's built from the song folders. The first time I run it in any given 12-hour-ish period, even with no changes to the song folder, it gives me The Long Load Of Death -- where, yes, it seems like it's stalling for multiple seconds on certain songs, always the same ones. I'll see if I can set up a screen recording or something to catch which ones it is. After that's done, subsequent launches are near-instant...until StepMania decides that it's tired of being fast and goes through The Long Load again. Really don't know what's going on there.

10-17-2016, 12:15 AM	#11
MinaciousGrace FFR Player Join Date: Dec 2007 Location: nima Posts: 4,278	Re: [SM5] xwidghet's Custom Build the game for whatever reason wasn't writing entries into an index file as it cached each song (which means interrupting the song caching process at 99% would result in starting from scratch upon restarting, also technically it did, but only for banners and not songs?), xwidget fixed this along with making other substantial improvements to songload first time song load times should be cut by about 40% (numbers kinda wobble depending on whether or not you're running off a hard disk or ssd) and subsequent loads should also be slightly faster but that issue is separate from yours i guess; the cache files are located in save/cache, not song, so that's what you should be monitoring. The only thing I can think of is that your cache somehow gets deleted upon reboot or something, or permissions get megafukt and the game can't read the files??? But that wouldn't explain why it only happens with the custom build and not the base. None of the io functions to the file system were modified in any way apart from what was being written in them. If it's irritating enough a possible solution would be to keep a backup of your cache folder somewhere else on your hd and paste it back any time you're having the long load issue. That's as much as I can guess at without being able to reproduce the issue myself :/ Last edited by MinaciousGrace; 10-17-2016 at 12:17 AM..

10-17-2016, 07:32 PM	#12
TyroneSama FFR Player Join Date: Aug 2016 Posts: 8	Re: [SM5] xwidghet's Custom Build I'm looking back through earlier posts and I have no idea why I implied that the issue was specific to the custom build. It happens on vanilla 5.0.12 for me as well. Sorry for being a potato. In any case, the behavior I'm getting seems consistent with cache problems, but I can't figure out what causes the cache to get thrown out -- rebooting isn't what's triggering it, and none of the files in Roaming/Stepmania 5/Cache/Songs seem to be disappearing or getting overwritten (according to their last-modified date). Moved everything to my new SSD, so that hopefully rules out a failing or misbehaving drive, and I even tried deleting the cache manually so that SM could rebuild it; lo and behold, death load a few hours later. I didn't really mean for this to turn into an extended General Stepmania Tech Support exchange, so sorry for shitting up the thread. If there really are no loading changes in 5.1, I'll keep poking around for an explanation. Last edited by TyroneSama; 10-17-2016 at 07:34 PM..

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)