So, the consensus seems to be Governments would rather get low bids that don't price in unknowns. If unknowns happen, they can dicker over the increase. But if you award a contract that's too fat, it's difficult to claw back.
If the work goes exactly to plan, the contract will come in on budget. But if problems occur, the price usually goes up. Our current system seems to be optimized for lower overall project cost at completion, with the side effect of making it less useful as a budgeting tool.
They don't really want integrity, they want to get a number that won't get them yelled at by their boss. The integrity bit is a pass the buck clause.
So the estimator is in the position of trying as hard as they can to lower the estimate. "Well, if we assume xyz unreasonable assumption, then we can lower the amount by this amount."
And then there are "risk" shenanigans. You itemize all the risk items, declare an arbitrary probability of happening. Then management rolls those up, and declares that they are not going to fully fund the risk and instead "carry" it forward.
Fully funding basically would mean multiplying the likelihood of occurring by the cost, and adding all the items up. Which is already iffy as the probabilities aren't based on much except a w.a.g.
So you win the bid, the risks are realized, and weren't funded. blow out.
Standard practice. And because everyone does it, the honest bidder is the loser. Losers go out of business. So the whole bidding practice is self selecting for cheats.
If you say, "this is unethical, I'm going to do my best to give a realistic bid", you lose.
Austin government did this for its MoPac toll express lanes and got them for way cheaper than what they should have paid: https://www.bizjournals.com/austin/news/2017/09/27/big-settl...
However, the project ran over for two years longer than it should have (it was supposed to be completed in September 2015, and they are only just now wrapping it up). So I guess no one really won there aside from the Austin transportation budget guys, except the populace hates them now for traffic jamming the place up for two years longer than they said they would.
Let alone that we basically award this work one estimated number and the ability to get through a paper work hurdle.
If you start making those rules complex it becomes hard to compare bids to each other, since your bids are likely to become non-transitive based on the scenarios.
Cost plus is the normal way this is side stepped, but unless you have a system that is mostly ethical actors that can break down. And if you have all ethical actors pretty much any system will work...
Why not just have major contractors required to post a bond when they place a bid, and if they have cost overruns they pull those from the bond (ie, their own money)?
Case in point: SpaceX challenging the launch industry's cost-plus culture with publicly-posted fixed prices [1].
Case in point #0: the Space Shuttle (STS).
[1] https://www.nasa.gov/pdf/382045main_19%20-%2020090730.11.STS...
Cost plus percentage gives a very clear incentive to raise costs
Compare the original winning bid for the STS with the actual costs; I think they rose more than tenfold (and the per-flight cost rose more than that), IIRC.And had the competing team won, the Challenger disaster never would have happened, because the solid-rocket boosters would have been manufactured as units and transported to the Cape by barge rather than having to be broken into small enough components to fit on rail cars.
I am not sure if you can implement a system that would work well when your actors are profit driven work that needs to be done can only be done exclusively by one entity at a time, and as a bonus that extra profit gives you more feedback into how the system operates. There aren't a lot of natural negative feedback to keep things under control in that system.
Especially if your goals are to reduce overhead.
I'm criticizing a system that incentives optimistic initial bids that are later revised up. A bad system doesn't need bad actors to produce bad outcomes. A $275 million bid ballooning to $500 million without any competitive pressure is a bad outcome.
American infrastructure costs are higher than production factors, e.g. land, material and labour cost differentials, explain [1]. Our auction mechanics are called out by experts as a big part of the problem.
[1] http://marroninstitute.nyu.edu/content/blog/is-u.s.-infrastr...
If you want separate bidding for separate phases, bid it out that way.
If you now introduce scenarios where projects can have an unknown number of bidding processes all costs will go up to cover the work necessary for multiple companies to prepare bids.
It would just make things much much worse.
Our bid was 1/3rd of the "winning" bid.
The distrust of government and corporations in our society is one of the biggest problems facing us as we go forward in the 21st century. We need to figure out how to restore trust.
The fix is not clear. Can't just drop in agile, every attempt to legislate away corruption fails.
It seems unlikely that bad faith is the primary cause. Yes, there are really contractors and owners who do act in bad faith, like the Trump organization, but these are unusual, and highly publicized. Punishing dishonesty seems like the easiest problem to solve, and it's easy to rally around for a quick and cheap victory. But I have seen these run-up bids for decades and no amount of punishment has ever seen to stem the flow of these events; it seems as if every city and state is infected with this problem. Either we are in a system where bad apples are so numerous that punishing a few has no meaningful effect on incentives; or we are in a system where bad apples are so rare that we punish good apples mistakenly, and can't figure out the real cause for our poor planning. Until we move on from the assumption that a few bad apples are to blame, we likely won't be able to discover the real underlying problem.