Supporting your examples, in previous GIS work I have iterated through a series of public URLs with file ID integers in them (Corresponding with individual document IDs) to batch-mirror whole public GIS datasets from government websites. This was on a public server, no http authentication or username/password, just something that was presented with a terrible javascript user interface only designed to retrieve one file at a time, slowly. Exactly the sort of thing that got this guy arrested.