Past performance before a project start is the average of an organizations output in whatever estimation measurement is used (story points, function points, etc). Some of the most reliable up-front estimates are based on such past performance data. That the projects differ, does not prohibit extrapolation.

As a project gets underway, you get the best possible indicator, namely past performance for the specific project. If you do regular re-estimation, progress monitoring and risk management (and you should, if you are a professional PM), you might not know if you are 52 or 53% done, but you will definitely know if you are ~80% done or ~50% done.

If you want sources, read McConnell's "Software estimation" and references, or if you are in a hurry, take the bite-sized blog post about "Yesterdays weather" from Fowler: http://martinfowler.com/bliki/YesterdaysWeather.html which is based on the same experience and literature.

It really depends very much on the project or organisation. If you're doing anything that involves risk, creativity, innovation, or research, it's almost impossible to predict. And even projects which contain little or none of these aspects often seem to go way beyond their initial budgets.

Agreed. R&D is notoriously hard to estimate and high-risk projects require tight risk management to allow proper progress monitoring. And yes it also depend on the organization as most organizations don't even keep records of initial vs actual estimates, making estimation based on historical data hard to impossible.

As for projects going over initial budgets, that's a different problem from progress monitoring, though they often go hand in hand.