FWIW I think that's more the core count than anything. I have a M1 Max as a personal machine and an M3 Max at work and while the M3 Max is definitely faster, it isn't world-beating.
My current work machine is M1 Max 64Gb and it's the fastest computer I've ever used. Watching rust code compile makes me laugh out loud it's so quick. Really curious what the newer ones are like, but tbh I don't feel any pressure to upgrade (could just be blissfully ignorant).
I very much enjoy being able to start compilation and just seeing results fly by.
My work machine was upgraded from an M1 with 16GB of RAM to an M3 Max with 36GB and the difference in Xcode compile times is beyond belief: I went from something like 1-2 minutes to 15-20 seconds.
Obviously if opening a browser is the most taxing thing your machine is doing the difference will be minimal. But video or music editing, application-compiling and other intensive tasks, then the upgrade is PHENOMENAL.