News for hackers. Stuff that matters.
Re:“Stop reverse engineering our code” by discreditable on Tuesday August 11, 2015 @08:02AM (#10040346)
Link is giving me a 404. Anyone got a mirror?
Re:“Stop reverse engineering our code” by anglebracket on Tuesday August 11, 2015 @08:04AM (#10040354)
Re:“Stop reverse engineering our code” by opless on Tuesday August 11, 2015 @08:21AM (#10040423)
Just in case a robot.txt kills that

http://pastebin.com/rcPSyRnR

Re:“Stop reverse engineering our code” by snsr on Tuesday August 11, 2015 @08:27AM (#10040456)
It's also on seclists.org -

http://seclists.org/isn/2015/Aug/4

Re:“Stop reverse engineering our code” by hughw on Tuesday August 11, 2015 @10:40AM (#10041376)
Would archive.org typically honor a robots.txt for a resource it already retrieved? I never understood the intent of a robots.txt to be retroactive.
Re:“Stop reverse engineering our code” by mikeash on Tuesday August 11, 2015 @01:46PM (#10042985)
Apparently yes, it would: https://archive.org/about/exclude.php
Re:“Stop reverse engineering our code” by syncsynchalt on Tuesday August 11, 2015 @05:23PM (#10044493)
My understanding is that sites like archive.org honor robots.txt retroactively not because they are required to, but to best honor the wishes of the content provider.
Re:“Stop reverse engineering our code” by X-Istence on Tuesday August 11, 2015 @03:50PM (#10043843)
Yes, it simply hides the content, it is still kept in their database so if the robots.txt disappears, it pops back from their archive.

New pages won't be archived though.