In theory anything could be in ram could also end up in swap. So yes, it is important. And yes, there is overhead, but it isn't too bad if your system isn't RAM constrained, and your CPU supports AES-NI or similar.