It's fine if stale data is used deliberately, problems arise when stale data is assumed to be 'fresh'.
A nicer way to put it might be that finite TTLs should be viewed as an opportunity to optimize, not ideal or standard.
However, just because it's hard it doesn't mean we shouldn't do it the right way.
Wouldn't deploying a less-than-precise TTLs be an appropriate trade-off for non-transactional caches, caches subject to network partitions, caches that can't enroll in invalidation messages, caches that can't poll for change sets, caches that can't implement eviction policy, etc?
Certainly the cache should be transparent about the trade-offs it has made (such as not promising authoritative data if it's accepting eventual consistency via TTL).
It's impossible to create perfect systems, so it always makes sense to give yourself an extra out to protect you from unanticipated defects (otherwise known as a "belt and suspenders" approach).