I want to use the FFR API, but I suck at web coding

[Trumpet63]

I asked my friend who knows a lot more about web coding things than I do, and he says:

1. I need a webserver to make an http request

2. The best option is Apache, but it's a nightmare to set up.

Now, I think I know what a webserver is, but not really what an http request is. And I assume I'm going to have to use php somewhere, but really all I'm trying to do is some math operations on the levelranks. Something simple like this page.

To be clear though, I don't necessarily care if individuals can access the page at this point, I know that requires having a web host, and money. I just want something like a file I can open with my browser, or if it has to be an .exe, that's fine too.

Umm... help please?

[rushyrulz]

correct me if I'm wrong, but the FFR API doesn't allow you to create web pages on FFR, it just allows you to access some backend information that usually wouldn't be accessible without logging in, which for now, only includes ranks. If you're interested in making your own webpages, you'd need to get a domain name and set up an apache server, among other things.

As for creating a program to do operations on this data, it would be helpful to know what kind of programming experience you have. It's relatively simple to do this kind of stuff in python/ruby/perl or other languages that use a regular expression engine for data extraction, but it's obviously a long road ahead if you don't have any prior experience with scripting languages, or especially programming in general.

[Trumpet63]

Originally Posted by rushyrulz

correct me if I'm wrong, but the FFR API doesn't allow you to create web pages on FFR, it just allows you to access some backend information that usually wouldn't be accessible without logging in, which for now, only includes ranks. If you're interested in making your own webpages, you'd need to get a domain name and set up an apache server, among other things.

Making my own webpage is exactly what I don't want to do... err, well, I don't plan on getting a domain name, but I want to accomplish roughly the same thing as a webpage, that is, having some way to make and run a statistics "program", and some way to share it. A webpage is optimal for sharing, but without paying for a domain, there's got to be a better way to share that doesn't require people to copy and paste in their stats!

As for creating a program to do operations on this data, it would be helpful to know what kind of programming experience you have. It's relatively simple to do this kind of stuff in python/ruby/perl or other languages that use a regular expression engine for data extraction, but it's obviously a long road ahead if you don't have any prior experience with scripting languages, or especially programming in general.

I have a lot of experience with programming in TI-BASIC (lol), and a bit of experience using KhanAcademy's javascript coding applet. I don't care about having to learn a new language, since I'd mostly be doing math and table operations the syntax can't be too bad to learn. It's all the software and prep you have to do before that that's killing me.

I need help with the [FFR API <=> Some Program] step, and then whatever steps you have to take to get the "Some Program" environment working (although I might be able to follow a wiki for that part).

I can't be the only one who wants to do levelranks stats, so maybe others can get help too? Unless all the cool kids already have a website >_>

[rushyrulz]

There are usually modules inside of these languages that allow you to scrape webpages.

[Trumpet63]

Originally Posted by rushyrulz

There are usually modules inside of these languages that allow you to scrape webpages.

Okay, so, describe how it's done. I don't have a preference for a language.

[rushyrulz]

sure give me a sec I'll code up an example.

EDIT:

Here's a short program that might look a bit complicated at first glance that takes info from FFR API and simply prints out a list of all the songs:

perl code

1  #!/usr/bin/perl
2 
3  use Modern::Perl;
4  use WWW::Mechanize;
5  
6  my $url = 'http://www.flashflashrevolution.com/api/api.php?key=MYSUPERSECRETAPIKEYGOESHERE&action=ranks&username=rushyrulz';
7  my $m = WWW::Mechanize->new();
8  $m->get($url);
9  my $page = $m->content;
10  
11 my @records = ($page =~ m/"\d+":{"info".*?"rank":\d+}},?/g);
12 
13 for (my $x=0; $x < @records; $x++){
14 	if($records[$x] =~ m/"name":"(.*?)"/g){
15 		say $1;
16 	}
17 }

the WWW::Mechanize module allows me to grab the page from the API and I store the content in the scalar variable "$page" (lines 6-9) I then use regular expressions to separate each record and store each of them into an element of array "@records" (line 11). Then I grab the song names from the data and then print them out on lines 13-17

output:
(obviously goes further up to include the rest of the songs)

[Trumpet63]

Originally Posted by rushyrulz

Here's a short program that might look a bit complicated at first glance that takes info from FFR API and simply prints out a list of all the songs:

*Mind Explodes*

Actually that doesn't look too bad. Thank you for doing that up for me.

Can you direct me to your preference of compiler and IDE, if they exist? (And anything else I should know before I try this?)

[rushyrulz]

http://padre.perlide.org/

Padre is a pretty great Perl IDE.

Resource for learning perl: http://www.perlmonks.org/?node=Tutorials

[Trumpet63]

I was about to freak out when I thought I had to install the module for Mechanize, which is apparently an immense task that is impossible to understand... but it turned out to just be a syntax error.

[rushyrulz]

installing modules is quite easy with cpan.

[Trumpet63]

Originally Posted by rushyrulz

installing modules is quite easy with cpan.

That's what it was telling me... but I didn't get it O_o

Anyways, update...

I'm trying to understand this "regex" stuff, so far I understand the $page =~ m/"\d+" part.

11 my @records = ($page =~ m/"\d+":{"info".*?"rank":\d+}},?/g);
12 
13 for (my $x=0; $x < @records; $x++){
14 	if($records[$x] =~ m/"name":"(.*?)"/g){
15 		say $1;
16 	}
17 }

We're matching the contents of $page with... something

m/ = a "delimiter"
"\d+" = one or more repetition of..?

:{"info" = first piece of string
. = concatenate
*? = any stuff
"rank": = second piece of string
/d+ = one or more repetition of
}}, = third piece of string

? = err... idk

/g = find global match...?

Mostly right so far?

Aaaand I'll just put this here. This is what the output from the API looks like, where X is relevant data from the levelranks:

{"score":X,"perfect":X,"average":X,"miss":X,"boo":X,"combo":X,"played":X,"timestamp":X,"rank":X}},"X":{"info":{"level":X,"genre":X,"name":"X","difficulty":X,"notes":X,"length":"X"},"scores":

[rushyrulz]

\d = match a single digit
\D = match anything but a digit
\s = match a single whitespace character (usually space)
\S = match a single non-whitespace character, anything goes
\w = match a single word character, letter
\W = match a single non-word character
\b = match a word boundary (either the beginning or end of a word)
. = wildcard, match anything except a new line character

modifiers to be appended to the above:
+ = one or more
* = zero or more
? = zero or one

gonna explain the regex I used.

my @records = ($page =~ m/"\d+":{"info".*?"rank":\d+}},?/g);

general what's happening:
@records is a blank array so far and $page contains the entire API ranks webpage. m/REGEX/ means I'm going to try to match something. The /g means that I'm going to try to match it as many times as possible. Each time a new match is made, store it as an element of @records.

The record format that I want:

"1":{"info":{"level":1,"genre":13,"name":"Terror From Beyond","difficulty":8,"notes":514,"length":"2:50"},"scores":
{"score":796700,"perfect":514,"average":0,"miss":0,"boo":0,"combo":514,"played":44,"timestamp":1288212311,"rank":1}},

regex explanation:
" plain text, match the string, which is just the double quote character, this marks the beginning of the section I want.
\d+ match one or more digits.
":{"info" match the plain text.
.*? This one's tricky. The best way to explain is with an example. Let's say we're given an input string, "abcbabcba". If the regex, m/a.*c/ is used, it will take as much as it possibly can and include it in the match, so it would be "abcbabc". adding a ? to the end of a .* (making it m/a.*?c/) makes it a non-greedy match which makes it take the first possible occurrence, meaning "abc" would be matched. Since the API ranks page seems to be just one giant continuous line of text, using .*? is necessary, otherwise it would match the entire page up until the rest of the pattern is recognized at the very end.
"rank": plain text
\d+ one or more digits
}} plain text
,? optional comma. There is a comma on every song except the very last one, for obvious reasons, making the ? necessary.

While we're at it, this deserves an explanation as well:

13 for (my $x=0; $x < @records; $x++){
14 	if($records[$x] =~ m/"name":"(.*?)"/g){
15 		say $1;
16 	}
17 }

If you're unfamiliar with c-style for-loop syntax, learn it, practically every language nowadays uses it.
One thing that might throw you off right away is the condition section of the loop, "$x < @records". One of the beautiful things about Perl is that it will figure out what you're trying to do based on contexts. By comparing the scalar, x, to array, records, the compiler will use the amount of elements in @records instead of its contents. Neat huh? It's the same as using an array.length() function in C or Java.

Line 14. The regex used is much simpler this time, but it does have a new concept. Basically, the regex is checking if the current element matches the form "name":"COULDBELITERALLYANYTHING", keeping in mind that a non-greedy match is used via .*?. Since the purpose of this program is to grab the song name, which could be anything from numbers to backslashes to 8 word phrases, .*? is definitely needed. By now, you've noticed that there is a set of parenthesis around the .*?. A programmer who has never used regexes before would just say oh yeah, that's just some extra grouping for the sake of neatness, and while parenthesis can be used for that purpose, it's doing a bit more here... Using parenthesis inside a regex will actually assign that specific matched content into a reserved scalar variable, $1, this is known as back-referencing. You can then assign that value to a different user-defined variable if you would like, but for this program's purpose it isn't needed.

since I'm using the "Modern::Perl" module (which you should use in every perl program ever, since it provides warnings and extra shit like that), I can use the "say" function, which acts as a "System.out.println()" in Java, that is to print the content to the terminal and make a new line. It's odd how that type of functionality isn't built into Perl's normal function library, but what can ya do?


Hope this helped you understand a bit what's going on. A great resource for practicing regexes can be found here: http://regexr.com. You can put whatever you want into the text field and play around with regexes to see if you can learn the ropes a bit. The tool also has back-referencing capabilities so you can mess around with that a bit too.