For something similar http://www.ebay.com/itm/Sun-SunFire-X4600-M2-8x-AMD-Opteron-...
The only reason to use hosting providers is to provide instant provisioning/scaling. Otherwise, money burned. Colo space is fairly cheap, so are old but still totally serviceable bad-ass machines.
I'm not sure you have the same definition of "cheap" as I do. $5/mo for a 1GB DigitalOcean droplet is cheap.
Buying old-but-badass hardware seems to be similarly cost-effective for that purpose (if that link is to be believed), but where (other than "a friend in ops") can you get 1U/2U of colo hosting for anywhere near that price?
I guess, if you're talking about the marginal price of an extra 1U/2U on top of your existing production rack, it can be negligible. For non-business-affiliated goofing-around, though?
Both are "cheap" just along different axis.
...when you say 'dev server', is that a computer in your home/office, or something hosted somewhere? And, if it's hosted--who are you using for hosting that makes 128GB-memory machines available cheaply enough that it's at-all-sensible to use one for a dev server?