I'm a heavy Gentoo user and a weekend should be enough. Especially if you've compiled a kernel before (do this btw, don't use genkernel). As for speed, I doubt you'll really doubt you'll notice much. Memory usage and installation size is way more noticable. If you need a speed boost I suggest not using a Desktop Environment, but using a simple lightweight window environment (I use Awesome, some friends of mine use i3) in combination with xfce4 utilities, since they don't have a gigantic amount of dependencies and are very nice to use.

My dotfiles repository also contains my gentoo config files: https://github.com/JelteF/dotfiles/tree/master/etc/portage Don't copy paste all of it, but it might be nice to see what a config aquired over a couple of years looks like. And some stuff is just nice to have for a desktop/laptop that you regularly use.