Bot that plays NES games by making numbers go higher

[Patashu]

http://www.cs.cmu.edu/~tom7/mario/

Ok, this isn't my work but it's really, really cool. Basically, this guy wrote a bot that plays NES games by attempting to increase addresses in memory that have a lexicographical ordering (for instance, score goes up but it has more than one byte - but lexicographically it still increases when you observe someone play) and trying multiple future sequences and greedily picking the best one. Without knowing anything about a NES game except what it sees happen in memory from the game being played once by a human (which it uses to decide what memory addresses are important, e.g. increase lexicographically), it displays emergent skill in games like Super Mario Bros due to being driven to increase score and scroll the camera right (two lexicographical orderings) and therefore to progress and not to die to hazards (preferring to get their score). It even finds glitches in the game and does insanely risky dodges purely because it keeps the bot alive.

In the video the gameplay starts at about 6 minutes in. He's published a paper and source code for it.

While the bot only gets so far before dying in each game, I think that if it was made more 'death-averse' somehow and looked a bit further ahead it would be able to beat simple NES platformers. It's really exciting that it can get so far with so little knowledge of the game![/i]

[Hakulyte]

Pause is best strategy ever to not die in Tetris~ that was a fun watch. That looks like an amazing tool to find glitches btw.

[Crazyjayde]

The man himself is really interesting to listen to. This is a great push in an interesting gaming direction, and I feel like this program could be exploited to explore new game mechanics.
Impatiently waiting for the day where TAS players will be replaced by bots.

[darklordsarumon9]

Stomping Goombas from the bottom, that's great.
This is an awesome video

[PrawnSkunk]

I'm curious as to what would happen and what limitations there would be if this bot were to "learn" FFR/Stepmania.

[justin_ator]

This was very intriguing

[Izzy]

It seems like it could be infinitely good if you had a collection of input data from thousands of people across every level.

[Untimely Friction]

Originally Posted by PrawnSkunk

I'm curious as to what would happen and what limitations there would be if this bot were to "learn" FFR/Stepmania.

Autoplay.

[cedolad]

lol rage quit

[Oni-Paranoia]

Originally Posted by Untimely Friction

Autoplay.

A learning program would have a curve before hitting all the notes perfectly. I could imagine a program getting a boo or good as the first judgement and then trying to replicate a bunch of goods before it notices perfects give you more points (assuming you didn't play beforehand, otherwise it would probably just look like autoplay, but that's no fun).

Very interesting video indeed.

[ReikonKeiri]

as a computer programmer nearing graduation my mind has just been completely blown. i feel retarded

[arcnmx]

Damn, it's just so clever and simple. Great use of the emulator.

Originally Posted by PrawnSkunk

I'm curious as to what would happen and what limitations there would be if this bot were to "learn" FFR/Stepmania.

FFR/SM is so simple it would be able to autoplay instantly. Since the bot doesn't really learn how to play the game (essentially it just tries playing it quadrazillions of different ways every second, checking each run in the emulator and picks the sequence with the best score)... As long as it can find the score, know which 4 buttons to press and looks ahead farther than the ~8 frames of timing window in a rhythm game it would never boo, miss, or do anything but perfectly hit everything.

[Izzy]

I think this bot would hit tons of boos on ffr just because it makes the score go up, but would only hit enough to prevent it from dieing.

Edit: oh nvm boos don't make your score go up. But goods in stepmania do despite killing your combo.

[Reincarnate]

This is the coolest thing I have ever seen


edit: Wouldn't be hard to apply to rhythm games. Keep your combo going up, keep perfect count going up.

[Ohaider]

This is awesome haha

[igotrhythm]

I can see it doing well at the original Mega Man because that had a scoring system and linear levels. Anything past that, though, it's going to struggle mightily because there's nothing telling it what constitutes progress through a level other than survival. And then there's the boss fights--the Karate Kid demonstration proves that lexicographical tracking is not the way to go when life bars are involved.

Just wait until this thing gets unleashed on Robot Unicorn Attack. O_O

[qqwref]

Despite its shortcomings this is a pretty solid achievement, and the video was really fun too. I'm surprised it already learned some conceptually tricky strategies (physics abuse, walljumping, ragequits). I think that although it isn't well-suited for some types of games, with some modifications, it could get really good. For instance, in Super Mario Bros. it could be specifically told that the number of lives may never ever be allowed to decrease, or in Tetris it could be told that pausing is illegal (and trained for long enough that it realizes lines give more points and are more important than anything else).

[j-rodd123]

i am not impressed

[mrpreggers]

i chuckled pretty hard at tetris.. pausing because it refuses to lose lol

[dAnceguy117]

Pretty incredible how the exact same program will play any NES game and, in some cases, play them very well. This guy rules.

How "greedy" is the algorithm, though, if it peeks into the future? Might as well grow those sequences and make more comprehensive choices. Less elegant, but more effective, right?

That juke to avoid dying in Pac-man was out of this world