1. Resolve hostname and remember the response
2. Verify that the response does not contain any addresses in a private IP space, or any other IP that is only accessible to you
3. Use the IP from step 1 when establishing a connection
With other solutions, you might end up being vulnerable to DNS rebinding attacks.
Bonus points for doing all your URL fetching in some sort of sandbox that enforces these access rules.
Remember that even ELB's in AWS have IP's that change all the time, and this itself is actually a source of vulnerabilities from apps that don't respect DNS TTL's (as has been seen in the forums repeatedly -- apps get connected to the previous IP instead of the new one). It's probably safer to retrieve and verify the IP for each request, and just cache if the IP is 'safe'. (And just doing IP subnet calculations is non-trivial in most less-common languages.)
Also, request throttling should be maintained and HTTP verb checking, to prevent being turned into a proxy for other attacks.
Actually, any decision to accept an arbitrary URL should be carefully examined in light of how hard it is to do safely.
So yes, there are probably lots of services which get this wrong; but they did at least get a warning about that particular failure mode.
We redid all our policies to be extremely restrictive in response, if the instance did anything based on user input. Anything more admin happened on a different machine.
Pentesters have been using this trick to pivot from unexpected backend web proxies to (e.g.) management consoles, LOM servers, JBoss interfaces, &c for 15 years or so already.