This is definitely not true across the board. I created a video player for reddit (http://dailyblocks.com) a while back which is completely client side rendered. Having done it this way I didn't need to setup or look for any server side technology. Simply put it up on any host that can serve up an HTML and JS file and be done with it.
> Single page and script heavy applications are more complex and take more time to develop and debug than server side rendered applications.
This also wildly varies depending on skill set and circumstances. Being a completely client side app all of the debugging tools come for free in my browser. If you happen to be more familiar with a different set of debugging tools then it's possible client side code takes more time for you to debug, but that's hardly a universal statement.
> They often duplicate logic between the client and the server.
Not true for isomorphic web apps. But again, depending on skill set this may or may not be more time consuming for you personally.
Now, there are cases for each. Building a simple blog, you'd be right: a server-rendered app with a sprinkling of JS for flavor would probably be a lot easier. On the other hand, a site that behaves more like an app might be quicker to build as an SPA. For example, an e-commerce site with complex filtering (you can filter by size, but it goes 0-20 for shoes, sm-xl for t-shirts, 10-70 for jeans, etc) and complex UI interactions available in-page on each item such as one-click buying, searching for similar items, setting up a price-alert, etc). You could do that mostly server-side, but I'd argue your frontend (at least on the item-browsing page) is heavier enough that it may be easier to just build a js-first site to start with.
This may all be wrong at each step, but hopefully it illuminates a bit how a lot of us (otherwise sensible developers) ended up thinking it makes more sense to start most things as rich-front-end+API apps.
Unless you are entirely using third party services, your single page application is probably contacting at least one service you also have to write. Now you have 2 applications to worry about, not just one. It adds complexity to your development, build, and release process. That complexity might be worth it.
Progressive enhancement leads to less initial effort. Single page and script heavy applications are more complex and take more time to develop and debug than server side rendered applications. They often duplicate logic between the client and the server. Alternatively, throw together some database queries and hook them up to html templates, and you have the foundation for a progressively enhancing site. You can then expend the extra effort to ENHANCE your site by sprinkling in client side code and cutting edge css features.