A minor tweak you can do is to exclude specific folders from the stuff to backup, but it'd be nice if the programs creating the caches would mark them as "no need to backup" automatically (which I think is possible?).

Still, I've got three million files on my computer - that's a lot of stuff to walk through. Looking forward to the APFS based replacement!