Batch Search Engine

[DossarLX ODI]

Last week hi19hi19 proposed the idea of introducing more automation to the Batch. This led me into taking a previous Search Engine I made for packs and difficulty search for pad files and turning it into a tool for the Batch that could potentially help improve the Batch process in the future.

You can try the live demo with this link, but be warned that cloud9 has a hibernate policy for servers that run for a while so the above link might not work if I've been running the Search app for a while. I wouldn't mind paying for several months to keep this app up and running without worrying about hibernate mode, but I want to convert this project to use SQL/Relational Databases in addition to integrating it on the FFR site itself, which will require me to talk to the Development Team. That plan will come at a later time; for now, this thread is meant to brainstorm what will benefit stepartists.

If the link is down, please let me know so I can restart the server. This Search Engine app runs on node javascript.

[DossarLX ODI]

Description
As a user, I would like to be able to use a search engine to figure out the status of a file I have submitted to the Batch.

User Experience
- The user is able to search by song name.
- The user is able to search by stepartist.
- Results for a search will appear in a table. The most important fields to display here vs. what is displayed on the Song Info page is still yet to be determined, there needs to be a balance between providing information and having too much clutter.
- Other relevant data not displayed in the table will be displayed in the Song Info page. These extra fields to display are still yet to be determined and what content they would contain, such as a Permission field.

Suggestions
- Extra fields: Length, Notes, and Individual Rating for the stepfile displayed on the Song Information Page.
- Average Rating for a Batch and Standard Deviation Information displayed on a separate page.

[psychoangel691]

This looks really nice

[blanky!]

[9:51:26 PM] blanky: @dossar would additional statistics about batch charts be added sometime to the engine
[9:51:36 PM] blanky: or would that give too much info

[FFR Pro 21]

Awesome!

[DossarLX ODI]

Originally Posted by blanky!

[9:51:26 PM] blanky: @dossar would additional statistics about batch charts be added sometime to the engine
[9:51:36 PM] blanky: or would that give too much info

Are you talking about things like average rating for a batch and then individual ratings for each song? That can be useful information for the song info pages, although probably would be too much in a results table in the search pages.

[blanky!]

I was talking about information about the chart itself (number of arrows, length, etc.) but what you mentioned would also be cool.

[hi19hi19]

Originally Posted by blanky!

I was talking about information about the chart itself (number of arrows, length, etc.) but what you mentioned would also be cool.

I don't see a reason why you couldn't include these things.
I guess if someone wants to keep number of arrows or length a secret?
I've never had occasion to do so, even with files such as Excite Bike where some of those stats were exceptional in some way.

[HeZe]

Looks nice! Unbelievable how quickly you did this

[XelNya]

Ok ok

this is fuckin' awesome

[ItsOnlyDanO]

now this is the kinda stuff we want to see

[Charu]

Song Nekuro no Makkuro Kuro Majutsu
Artist paraoka
Stepartist Charu & Gradiant
Status unjudged
Update Dec 29, 3:03
Batch November 2014

mfw

[gold stinger]

Are you telling me there's over 200 songs stuck in batch

I wouldn't know I never monitor this stuff

That would probably explain why it's such a big deal

[DossarLX ODI]

Originally Posted by gold stinger

Are you telling me there's over 200 songs stuck in batch

I wouldn't know I never monitor this stuff

That would probably explain why it's such a big deal

September 2014 Set1 - 23 Songs
September 2014 Set2 - 23 Songs
September 2014 Set3 - 22 Songs
November 2014 - 52 Songs
January 2015 - 51 Songs
March 2015 - 20 Songs
May 2015 - 16 Songs
July 2015 - 27 Songs

The Backlog has a total of 234 songs to be judged as of right now. With September 2014 completed it would still be 166 songs, and the Burndown Chart for the Batch would look atrocious. This is why I've been considering alternate methods for the judgment process since the current system won't be used after the September 2014 Batch is over.

For another update, I'll be posting the currently qualified submissions for the Difficulty 89+ Special Batch in the Queue/Batch thread tonight. This Special Batch is planned to be opened end of this week and will be the next batch of songs to go through judgment. If I recall there are 16 files that fit under this category, and that songlist will allow stepartists to replace these 89+ difficulty submissions in the other batches with other files and then move those hard files to the Special Batch. Details on this are still in progress. Any submissions moved to this Special Batch will have the original submission's batch field replaced with the Special Batch and open up another slot to be replaced.

[SoFast]

Nice

[Lambdadelta]

This is really great, I'm happy to just see where exactly I am in the batch.

[DossarLX ODI]

I just restarted the server to reflect the current files that qualify for the High FGO Special Batch. The breakdown is now this:

September 2014 Set1 - 23 Songs
September 2014 Set2 - 23 Songs
September 2014 Set3 - 22 Songs
High FGO Special Batch - 15 Songs
November 2014 - 49 Songs
January 2015 - 47 Songs
March 2015 - 18 Songs
May 2015 - 10 Songs
July 2015 - 27 Songs

In addition I'll be adding a suggestions section in the second post of this thread for what's been mentioned so far and any other ideas.

[M0nkeyz]

does ''update'' refer to the date it was submitted or the day it was updated? Because I've sent some fixes, but it doesn't show in the ''update'' row.

[DossarLX ODI]

Originally Posted by M0nkeyz

does ''update'' refer to the date it was submitted or the day it was updated? Because I've sent some fixes, but it doesn't show in the ''update'' row.

Having this Search Engine publicly available will allow stepartists to see if certain information is incorrect (such as the update field). You may very well have sent an update but it hasn't been seen yet. Update refers to the most recent fix submitted.

If anyone else sees information that is incorrect, please let me know so I can change it accordingly. I told blanky to enhance his NPS Generator by allowing it to export song length and notes data out to a file which I can put as more information for these submissions so stepartists can also see if the version we're using is the right length and has the right number of notes.

[DossarLX ODI]

I ran my batch parser through the batches again and found some issues:
- Monsoon in the Concrete Jungle from January 2015 is missing a .sm file. The sound file is there with a black image and nothing else. Who is the Song Artist, and can this be resent with an .sm file please?
- Glass Lattice from March 2015 is missing the Song Artist in the .sm file. Is the artist "Zircon"? Also please refrain from sending large wav files; this wav file is 26 MB.
- Her Signals is from March 2015 missing the Song Artist in the .sm file. Is the artist "Receptor"?
- Theory of Everything 2 from November 2014 is missing the Song Artist in the .sm file. Who is the Song Artist?
- Ashes to Ashes from July 2015 is missing the Song Artist in the .sm file. Is the artist "Zircon"?
- Scrambled Eggman from July 2015 is missing the Song Artist in the .sm file. Is the artist "BrainCells, SnappleMan"?
- Acrophobia from July 2015 is missing an .sm file.

In addition, ShutEmDown, Time Bomb, Straight to Video v2, and didn't have song artists specified.