Ok, this isn't my work but it's really, really cool. Basically, this guy wrote a bot that plays NES games by attempting to increase addresses in memory that have a lexicographical ordering (for instance, score goes up but it has more than one byte - but lexicographically it still increases when you observe someone play) and trying multiple future sequences and greedily picking the best one. Without knowing anything about a NES game except what it sees happen in memory from the game being played once by a human (which it uses to decide what memory addresses are important, e.g. increase lexicographically), it displays emergent skill in games like Super Mario Bros due to being driven to increase score and scroll the camera right (two lexicographical orderings) and therefore to progress and not to die to hazards (preferring to get their score). It even finds glitches in the game and does insanely risky dodges purely because it keeps the bot alive.
In the video the gameplay starts at about 6 minutes in. He's published a paper and source code for it.
While the bot only gets so far before dying in each game, I think that if it was made more 'death-averse' somehow and looked a bit further ahead it would be able to beat simple NES platformers. It's really exciting that it can get so far with so little knowledge of the game![/i]
Comment