The swap file stores edits rather than the whole file. I guess this could be laggy if you were doing global substitutions on a massive log file, say, but I've never had an issue. Vim was written for much slower computers.