Because it's not just about whether data is collected, or what is collected; it's about why it's collected. There are a lot of services for which I'd be willing to give every kind of tracking access short of a remote desktop connection, if and only if I was 100% sure it'll be used only for debugging and usability testing. But I don't want a single bit of my data to reach the marketing team, the third-party advertisers, or the sales people. Since I can't be sure, I'm not giving away my data.

Also, yes, you could argue that analytics for e.g. A/B tests are actually "improving the website/service". Yet I'd still prefer to opt out of them, because those A/B tests usually encourage making the software more addictive (erm, "engaging") and less useful ("only single-digit-% of users use this feature, so let's drop it"). I don't trust the motivations of those companies, so I don't believe they'll use A/B tests for my benefit as a user/customer.