"Yes it would be unfair to tax them at a higher rate for doing fundamentally the same thing - how would we justify one tax rate for one investor in a company, but another rate for this other investor?"

Because taxation has gone way beyond what it was initially meant to do. That's why it's a hard problem; we're trying to make value and fairness judgements and using that as a basis for applying tax.

To me, personally, people can't claim fairness if they're treating people differently based on their arbitrary value judgements. Until we start treating "fairness" as a universal, with no subjective component.