Initially thought just to probably be some numerical instability when the next actions are all equally likely (I.e. there’s nothing for it to do) but what was actually happening is the pseudo RNG was partially fed by controller input, and certain patterns of controller input would make random rewards more likely to spawn.
I always thought it was cool how these exploits get discovered by the deep nets
I was under the impression that machine learning is only good at learning things that can be approximated by continuous functions.
A RNG is almost the complete opposite of that. It's everywhere discontinuous!
Some games were worse, far worse. Doom used a list of 256 random numbers that it looped through.
Pokemon is another game where the RNG function is extensively mapped, and even used in combination with bugs to generate Mew events, something that should never ever happen.
You see a lot of TAS of old games, especially popular ones, that abuse the RNG generator where feasible.
Here's the link for anyone interested: http://www.roguelikeradio.com/2016/06/episode-122-nethack-to...
A very niche topic on an already niche podcast, but probably will get some decent overlap on HN.
(Also I'm assuming this is the same team? It's hard to tell.)
The bumping into a wall to advance the RNG state without changing the game time is a really neat trick that I would otherwise have been completely unaware of, and I have played Nethack for many years and done some source diving myself.
Truly a great feat of engineering here.