Sure that having more RAM won't make things slower. But for most uses cases SSD compensates for lack of RAM, as swapping becomes reasonably fast.
You need to swap our more than 200 MB per second to saturate SSD in MacBook Air, it is hard to produce that much of non-garbage data.
I have no idea why it takes so much memory and I consider it to be somewhat a problem. However, it's not a problem I can do anything about but I still have to live with it daily and get my job done.
Linux has excellent caching for I/O operations, so having more memory makes things faster, so even if something does not explicitly need the RAM, having it there will make thing s go faster.