Since you're a gentoo user, you probably already know that, but have you tried using compressed memory like zswap? I find that I very rarely swap out to disk since I enabled it (and tuned to my needs):

https://wiki.archlinux.org/index.php/Zswap