In fact, I just started a resource limited chess program competition: https://rlc-chess.com
If a site uses the loading=lazy attribute they won't even be loaded until they're on the screen.
Background images in in-line CSS can block rendering in some older browsers but that's really the url() bit that's blocking rather than the image.
This is a heavily optimised PNG favicon [0], which is only 192x192. But without trashing the palette, I can't really deduce it further, and it's already 2.7K. Niether pngcrush nor imageoptim can do much with it.
[0] https://shatterealm.netlify.app/android-chrome-192x192.png
And whilst the server might use state of the art compression... There's still no way a much larger image is going to fit in the proposed 1-2K limit.
It's more about the content of the image, the optimization can only do so much. If it's a pixel art or the palette is reduced to hell, then yeah, probably; otherwise I don't think you can display a 256x256 photo in 2KB.