Even with reasonable amounts of swapping you are unlikely to notice huge differences. Only if you are thrashing the memory, you surely would. But then just getting more memory would be a very wise investment.