Hell, they knew it from the PS1 era: Net Yaroze may not have stopped people from trying to hack their PlayStation 1, but it definitely appealed to a part of the crowd that would be most interested in that sort of thing. (And they even included a toolchain! Truly must've been special to be a part of that.)

I think the trouble is that people kept finding ways back from arbitrary code execution in a limited environment to running pirated games and cheating in online games, and Sony continually failed to actually manage that.