I dumped all of the non-important stuff (videos, music etc) into /Users/Shared and use Filevault only for dev and the size of it is fairly small. I haven't noticed any of the things you mentioned...