> The only uses I can think for it are nefarious, i.e. allowing outside services to track the user and violate their privacy.
A smartcard would be one of the worst or at least most complicated ways to implement tracking: It can communicate with the rest of the system only through an extremely limited interface and can strictly only ever answer requests sent by the host, never initiate requests on its own.
To do anything nefarious, it would need a privileged companion service on your computer – which doesn't gain anything from being able to talk to the smartcard.
As an aside: Even TPMs are an extremely passive technology. The only thing that arguably makes them "evil" is the fact that they can perform measurements for device attestation, but it can still never transmit these on its own. That evil is pretty indirect, in that some service providers might only allow users to use TPM-enabled and sufficiently attested clients to access their services, and exclude open hardware and software.
That's coincidentally exactly what DRM is, and it's already here, and not at all limited to TPMs. I'm cautiously optimistic though that it's possible to strike a compromise and limit attestation to properly sandboxed parts of the system, e.g. only the parts of the GPU relevant to display copyrighted movies, without getting undue access to the rest of the system.
The smartcard part of TPMs is about as capable of evil (as far as your computer and your data on it is concerned) as a USB-connected mug warmer.
You can use the "smartcard part" of a TPM. This gives you secure/non-extractable key storage.
You can use the attestation/trusted computing part of a TPM. This gives you trusted computing, which can be used for DRM, if you install software or use a service using DRM and grant it access to your system. If you don't like that, just don't do that. (Today's DRM solutions don't even use TPMs anymore, for what it's worth.)
If you want/need neither, just use neither!
The only think that kept DRM from leveraging it was indeed the low usage in consumer spaces.
You can kinda sorta do this with WebAuthn if the service you're enrolling into allows for multiple authenticators (the spec recommends this, but some services don't allow more than one). But then you have to repeat that enrollment step with all devices, for every new service you sign up to. Which is practically useless because an actual backup is supposed to be stored in a safe place that might be hard to get to.
> TPMs are essentially built-in smartcards
The question is why anyone sensible would want to have a smartcard built into their computing device. The only uses I can think for it are nefarious, i.e. allowing outside services to track the user and violate their privacy.