I'm not sure who you're talking to, or what kind of sites you are talking about; but I think my site is pretty fast loading: https://learntemail.sam.today/

One "trick" that I've found helps to add place-holder gradients for images. So if you have a background image, add a linear-gradient or something as well. That means that there isn't such a jarring transition between the gradient and the image.

But really - keep the JS to the minimum and you'll be good. Also try and avoid using a large CSS "framework" - as that can easily add more bloat. If you start from scratch and only add what you need - keeping it very lean.