By tracking you, they can remember that you are not a robot. As much as I dislike tracking, I don’t think having the captcha be always hard would be all that great either.

Ideally I’d like to see fewer captchas. But there’s no good alternative to it really. I mean, requiring phone verification instead is an alternative. But I don’t necessarily want to hand out my phone number to each and every site on the net that I interact with either.

Its absolutely not true that there is no alternative to tracking for captcha "hardness".

https://privacypass.github.io/

You can prove trust once and get several tokens to spend later. There are various people looking at this like Cloudflare and Google.