I'd think that was the salt of software architecture, but there are limits to which you can fix up a dish by adding salt, beyond which the dish becomes inedible. So I'd guess "upgrade hardware" would be the salt of software architecture.

That sounds like needless idealism

Do you really think your front page needs all the queries reran for every single user that visits? Even though things won't change for, let's say, even 30s?

Even the front page of HN is cached. Their system is pretty snappy but not caching what's trivial just doesn't work in the real world.

That suggests that I believe the right amount of ketchup on food is zero. That's incorrect. I'm not saying you shouldn't ever cache anything. I'm saying that caching is used to cover up a lot of architectural flaws, when sometimes we would be better off if we fixed the flaws.

And I'd add that you notion that the only two options are "cache X" and "rerun all queries" is a part of the problem here. There are more things, Horatio.

"There are only two hard things in Computer Science: cache invalidation and naming things."

Seriously don't build a cache until you have exhausted all other performance improving avenues. I am not against caches but throwing a new cache in the code every time a performance issue is found or worse preemptively making a cache due to a possible performance issue is an endless source of cache invalidation bugs.

Yes don't cache everything needlessly, use it wisely