Of course browsers have actually gotten pretty good at AJAX-like loading instead of completely re-rendering the page. These systems tend to rely on heuristics though, and I don't think there's any documentation for these, but even when these fail the browser tend to be more competent than even the best JavaScript solutions.
Your JavaScript shouldn't be blocking page load anyway. Defer, defer, defer.
https://blog.mozilla.org/javascript/2017/12/12/javascript-st...
https://bugs.webkit.org/show_bug.cgi?id=192782
Also did you mean http caching? Not sure why would want to remove that. It's still important for the browser to get the latest script content before the bytecode caching happens.
For example create a melody with seed: 4564342
The client can render it and if you access it from the server the server does the rendering with the same seed.
Caches also exists, now with PWA-s offline modes would benefit from the History API.
If you need an HTTP connection to download a section of HTML for a new part of an SPA it won't be that much different from a full page of HTML, presuming you compress the transfer as you should.
"Of course shitty implementations exist" is true of a non-SPA setup too.