The majority of the projects I work on target users on desktops with modern browsers. You have to get into the massive 10mb range before noticing any lag, and even then only on an empty cache. SEO isn't a concern either, all of our marketing pages (when we have them) are static sites.

It's all about picking the right tools for the job. If low latency is a business requirement by all means don't use a framework.