I'm worried that we'll never be able to build our own phones the way we can build our own PCs with commodity hardware due to the crazy patent situation.

Probably hardware and software will continue to be locked down (or in Android's case, released to open-source many many months after official release), but we'll be rescued by the web the same way we were in the PC era.