On the contrary, it works perfectly well for normal, non-bad actors running services used by most of the public. That includes search engines and stuff like archive.org. A robots.txt set to deny all will result in your site not showing up on any search engine that matters.

It doesn't work for bad actors, but then again, nothing really does.