News for hackers. Stuff that matters.
Re:NetHack beaten in 7 minutes 15 seconds real time by malux85 on Friday January 11, 2019 @01:33AM (#18881450)
This reminds me of something I read on reinforcement learning playing Atari / video games ... often you see the Deep net (I think the game in question was a tank game) just spazzing out on the controller when there’s no enemy, the tank was spinning on the spot while waiting for a door to open for example.

Initially thought just to probably be some numerical instability when the next actions are all equally likely (I.e. there’s nothing for it to do) but what was actually happening is the pseudo RNG was partially fed by controller input, and certain patterns of controller input would make random rewards more likely to spawn.

I always thought it was cool how these exploits get discovered by the deep nets

Re:NetHack beaten in 7 minutes 15 seconds real time by zawerf on Friday January 11, 2019 @02:56AM (#18881702)
Do you have a source on this?

I was under the impression that machine learning is only good at learning things that can be approximated by continuous functions.

A RNG is almost the complete opposite of that. It's everywhere discontinuous!

Re:NetHack beaten in 7 minutes 15 seconds real time by dklsafhjskljfl on Friday January 11, 2019 @03:08AM (#18881735)
Old games put the P in PRNG. Often, the technical limitations caused them to very roughly approximate randomness, for example a function that used the last n button presses plus the countdown timer as a seed. If your game has to run in under a kilobyte of memory, that n value might be a byte, or less (such as four bits, 2^4)

Some games were worse, far worse. Doom used a list of 256 random numbers that it looped through.

Pokemon is another game where the RNG function is extensively mapped, and even used in combination with bugs to generate Mew events, something that should never ever happen.

You see a lot of TAS of old games, especially popular ones, that abuse the RNG generator where feasible.

Re:NetHack beaten in 7 minutes 15 seconds real time by simonh on Friday January 11, 2019 @03:08AM (#18881736)
If the result from the RNG is not actually perfectly random, and is actually partially determined by e.g. controller input, then even if the function is discontinuous there may be a continuous function that approximates it closely enough to be useful.
Re:NetHack beaten in 7 minutes 15 seconds real time by dharmab on Friday January 11, 2019 @09:06AM (#18883621)
Old games have such predictable RNG that humans can manipulate them in non-TAS runs. See the run for Final Fantasy 1 for a good example.
Re:NetHack beaten in 7 minutes 15 seconds real time by bluGill on Friday January 11, 2019 @08:54AM (#18883562)
Atari computers actually had a hardware random number generator driven by electrical noise. (I'm not sure what the 2600 system had, most of their latter game consoles were based on their computers). Most other computers in those days just used a pseudo random number generator.
Re:NetHack beaten in 7 minutes 15 seconds real time by sehugg on Friday January 11, 2019 @10:43AM (#18884249)
The 2600 didn't have a built-in RNG, but many games took advantage of uninitialized registers on powerup to seed their own RNG.
Re:NetHack beaten in 7 minutes 15 seconds real time by karmakaze on Friday January 11, 2019 @12:27PM (#18885068)
I remember it as using a pseudorandom noise source. Wikipedia on POKEY[0] says 8-bit values from a 17-bit polynomial counter. Sounds about right.

[0] https://en.wikipedia.org/wiki/POKEY