The ones for < $890 are running NetBurst Xeons ... which means, at the youngest, they're nearly 4 1/2 years old.
The one at $890 uses a Xeon 5120 which at least gets you into the Core Xeon era, but you're still looking at a more than 4 year old processor.
If you are going to cheap-out, at least do yourself a favor and buy something low-end-but-new from somewhere like Dell. You can pick up a new PowerEdge T110 from Dell for $400 which comes with a only-released-a-year-ago Xeon X3430 which is going to be about 3 times faster than that Xeon 5120. Plus you're going to have a faster FSB, memory, drives, networking, modern interface to upgrade with, etc.
If you're looking at spending $1k/yr for colocating your server, make it worth it by co-loc-ing a server that's not 5 years old.
If you're OK with the servers being hosted in Germany, you can rent a quite powerful dedicated server from http://hetzner.de for roughly the same amount of money.
Look for the EQ4-10 servers (http://www.hetzner.de/en/hosting/produktmatrix/rootserver-pr...)
I've been with them for almost a year now and I can only recommend them.
You can buy a very cheap server that's much more capable than a cheap VPS server. Here are some decent servers for about $300; you can often find good servers for $150:
http://www.geeks.com/products_sc.asp?Cat=821
Colocation at a good facility costs $60-100.
So for $150-$300 upfront and $60-$100 monthly, you can have a capable server hosted at a good facility at a much cheaper cost than the equivalent "rented" server.
You can use a VPS or something like EC2 as a hot backup, setup replication, and have a very capable setup for not a lot of cost.
This assumes that you have sys admin skills.