For one, it uses onion routing, which means you are trusting that nodes are not colluding which each other to share information. In addition to the trusted watchtowers. If people start trusting the intermediate states of the payment channel then you could get into some really hairy territory in terms of trust.