Below constitutes my (limited) experience working as an engineer at a company with a big logistics component (not Amazon).
When you run an operation the size of Amazon (or even muuuuch smaller) and are putting out a lot of freight for UPS/FedEx, you batch up your outgoing material based on ship method and 'next hop'. In order to speed up the delivery from a 3-5 day window to a 2 day window, you would have to explicitly purchase the '2 day' shipping option for that customer, or it would get batched in with all of the other 3-5 day goods, which could get sent to a totally different distribution center than the 2 day.
We're talking huge numbers trucks all going to "Indianapolis 2 day" versus "Lexington Ground" or whatever - once it gets on that ground shipping truck, I don't think it matters a ton what your address is flagged, as it's going to go that shipping method.
The first and largest complaint people will make is that they can't use UPS Prime while on vacation or to ship gifts.