A single thread already has two different trust domains (user mode and kernel mode), so any pair of threads will have different trust domains when one of them is running a system call or an interrupt. So this solution would also have to either pause the SMT sibling while entering the kernel, or run the kernel code in a separate core (an idea like this was briefly suggested on the RISC-V isa-dev mailing list a few months ago: to have a separate core which handles interrupts and kernel mode for a group of user-mode-only cores).
If it was clearly outlined as 8 cores that support hyperthreads or 10 cores that don't for around the same price, what do we suppose most people would choose?
Part of the reason for the current status quo might be that hyperthreads are a big differentiator beteen AMD and Intel, and play towards Intel's strength (higher clock speeds at fewer cores).
We have something close to the proposed scenario starting to play out with AMD's offerings that support more cores and no hyperthreading, but it's not a perfect experiment because AMD's cores are also lower clock speed, and there's a lot of brand name loyalty currently.
Ah, that's what I was looking for. I'm aware that most the resources are shared, but I also assume there have been design choices in other components to make SMT easier or faster, and possibly that increases their die size a small amount, or in general just complicates their design.
The question (which is ultimately unanswerable) is what would we have had Intel not chosen SMT as the path to pursue? If they had instead invested those resource into other areas (e.g. more cores) and never let SMT concerns enter into the discussion, how would those (theoretical) CPUs compare?
That's the CPU comparison I was alluding to in the prior comment, and why I noted AMD vs Intel (even with all it's other differences) may be the closest match up of that idea we'll see.
It wasn't meant as a rebuttal to anything, it was more a "wondering out loud" type of comment spurred by yours, about what could have been had different paths been taken.
There's a very simple solution here. Don't schedule two threads with different trust domains on different SMT threads on the same core. No need to change any hardware, no need to disable anything, just accept (as has been known for at least a decade) that there is very likely to be a side channel attack if you look hard enough when SMT is involved.