On the Conservative side - many of the remain MPs come from leave constituencies. Also, as a party, the Conservatives have a strong leave bias. If the leave does not happen, it is hard to see the Tories making it back to power for a considerable time.