Do you really think your front page needs all the queries reran for every single user that visits? Even though things won't change for, let's say, even 30s?
Even the front page of HN is cached. Their system is pretty snappy but not caching what's trivial just doesn't work in the real world.
And I'd add that you notion that the only two options are "cache X" and "rerun all queries" is a part of the problem here. There are more things, Horatio.
Seriously don't build a cache until you have exhausted all other performance improving avenues. I am not against caches but throwing a new cache in the code every time a performance issue is found or worse preemptively making a cache due to a possible performance issue is an endless source of cache invalidation bugs.
Caches are global state in sheep’s clothing. In many languages and implementations, they can be mutable shared state, which leads to unexpected side effects and concurrency bugs that may cross transaction or user boundaries. It’s a covert channel where code interacts without ever talking to each other. In languages without static analysis you cannot determine quickly which pieces of code talk in this fashion. It’s all secretly coupled instead of overtly coupled. You only learn by vast experience, and in the interim you make mistakes, which makes the oldest team members appear capable and the newest unworthy of trust. But it’s the people who tilted this playing field that you can’t trust. In a word: toxic.
Done right, memoization shows up clearly in your sequence diagrams, near the heart of your architecture. The top level of the code knows what’s going on, and orchestrates communication. People walking in off the street can figure out what the hell is happening, and be productive in short order.
It can often force you to think about when your system of record and your source of truth diverge. Usually I see people cache low quality precursor data for a decision, and then they have to cook it every time.
The thing I liked most about AngularJS was that you were meant to fetch all of your data fairly early in an interaction, mis-en-place style. If the data you got was a little garbage, you cooked it before any interface code ever saw it. The view just saw the nice version.
Doing so affords another architectural tool. All of that cleanup code can be moved to the backend or the database in a single refactoring, getting it out of the client. And the overt coupling makes you contend with the fact that a piece of code talks to too many hints to get its job done. It’s a mirror some do t want to look in.
The problem is, all of this involves making decisions, and a lot of developers either never learned, or learned to distrust their decisions. And rather than finding way to mitigate a bad decision, far too many of us have instead spent years or decades learning how never to make a decision at all. We’d be a lot better off with strong opinions loosely held.
“But we have an architecture.” No, you don’t. You have a set of idioms that you arrange like dominoes until an answer you like comes out the other end. That’s only an architecture in the way suffering is a personality.
You don’t need more caches. You need an architecture and dynamic programming. Someone save me from promise caches. First couple times they were so cool, now they’re a golden hammer for lack of even the most rudimentary of data flow analysis.