The interface and outward appearance is simple, but the internals most certainly are not. There are still hundreds of components from dozens of manufacturers combined to make a physical product. In the case of the iPhone there's one of the largest mobile operating systems on there.

Computer chips are getting more and more complex as well, now with more levels of caching and inter-core communication.

I think we have a long way to go before we get simpler computers, but that does cause us to wonder if we will get to the point that they are just too complex.