Is there some specific knowledge that Pelosi allegedly traded on, that didn't belong to him?
Pelosi and indeed all of Congress has non-public information about laws, government contracts, etc. Right now they can and do trade with this "insider" information. Legally this is not considered insider trading. Congress wrote the legal definition of insider trading in such a way that excluded their insider knowledge. There is no value in arguing the semantics of this so I won't address it further.
The traditional solution for this glaring conflict of interest that is used for Presidents is a blind trust. Most Presidents put their wealth in a blind trust to avoid even the potential of a conflict of interest between their actions as President and their personal wealth. Trump did not do this and it was a shit storm. He even tried to get the government to rent out his resort for some UN thing. Presidents and Congress should all be required to put their wealth into a blind trust to avoid conflict of interest.
Blind trust is the only acceptable option. It's been tradition for Presidents through most of my lifetime. Trump didn't do it and it was rightfully a big shit storm.