(Like everyone else) Elance are out to make a dollar. The system isn't perfect, but show me a business that is.

I think it was HIGHLY optimistic to have a budget of less than $1,000 and to think things would be pain free.

It would have been interesting if the author had added what he had expected. The article is highly factual (this happened, and then this and then this...) but he didn't add any value judgment. What was his expectations? Was it a red flag that no one on Reddit wanted to touch it? Would he do it again? If he had twice the budget, would that have changed things? etc. etc.

I think you're on to something. Ratings on sites like Elance are broken. Clients feel compelled to give 5 star feedback to freelancers, sometimes just to get them off their back. This basically means that you can't look at a reputation score to tell you how good the freelancer actually is. I like the idea of a third-party anonymous person doing an audit. That person would need to have access to all the relevant files though, and have the right skills to judge the freelancer's work. Even the prospect of being audited could be a deterrent to the kind of behavior we saw in this case, which means that randomly auditing one out of every x gigs could be enough.

Real life jobs/recruiting solves this problem via references.

Why isn't this something elance can provide? You can get an anonymized contact info for previous clients of the dev. You can ping them for some questions etc.

Good idea - right now, the best proxy for this is to look at the % of clients who re-hire and their comments. While I agree that 5 star systems across the Internet are inflated, I always use rehire % and really scour the comments when hiring on Elance.

This is something that needs to be more widely understood. Direct feedback is gamed. Even Amazon reviews, which are more indirect than even professional reviews, are gamed.

It's a common response to the need for regulatory systems that we should "trust reputation systems" instead. This is what happens. Regulatory systems can at least be audited and held accountable. In reputation systems, absent an unforeseen innovation, you'll never be able to do better than currency.

That's how reputation has been used since the dawn of man. It has always been currency. Sites like Elance merely made efficient.