I dumped all of the non-important stuff (videos, music etc) into /Users/Shared and use Filevault only for dev and the size of it is fairly small. I haven't noticed any of the things you mentioned...
Leopard fixes this. When you log out, it "recovers" disk space and cleans itself up.
Actually, all versions of filevault have done this.
It requires far more HDD space than not using it because it doesn't seem to clean up after itself for a while. And if you get low on hard disk space your system grinds to a halt.