NVME over SATA drives will drastically improve file access times. You will find these on newer, pricier machines, but if your mobo has a slot, use it because the drives are fairly cheap.

Going from low clocked memory to high clocked memory can cost a bit of money (last I looked, it was like a 30-50% premium going from 2666 to 3200 to 3600MHz). As well, if you're comfortable, tightening the CAS timings on your memory can see noticeable improvement in memory bound applications. I personally have measured a 25% performance increase once my memory profile for 3200 was set correctly (mostly a Ryzen thing) and just upgraded to 3600 and haven't tested, but in my larger projects with tons of in-memory code I'm noticing improvements.

Iterating over a loop can be a world of difference depending on what is happening in the loop and what vector instructions your CPU supports, and how well it is supported. As well as your CPU's clock, L1/L2 cache sizes... basically everything.

I have used a computer with NVMe, higher clock speed, better caches, more RAM…the works (but the computers I’ve personally owned only have had some of those ;)). They’re faster, yes, but fractionally for short latencies. Typing a character on one and waiting for it to show up is not significantly better on the other.