I was under the impression that FPGA vendors often license functional blocks (like PCIe SERDES) to FPGA users. Might it be that part of the purpose of obscuring the bitstream format is to make it more difficult for customers to use those functional blocks without paying the toll?
Well, that's no surprise: FPGA vendors spend a lot of manpower on improving their Place&Route software. If you wanted to build something competitive, you'd need a lot of money plus access to proprietary, non-public, information.
Similarly, you can reduce the complexity of routing calculations by applying some constraints. You will potentially lose the possibility of an optimal solution, but you will gain a far faster compilation time. As always with engineering, it's a trade-off.
One of my favorite ideas on this is space-filling curves: http://www2.isye.gatech.edu/~jjb/mow/mow.pdf
In my opinion there is very little for the manufacturers to gain by keeping their bitstream formats proprietary and undocumented. I don't think there is a competitive advantage, as all the manufacturers are pretty much doing the same thing. And their FPGA block diagrams are already open and documented (you can see how many flip flops, clocks, and muxes are in each logic cell, how the routing works, and where the memory cells and other units are).