This is one of the reasons to keep the team as small as possible and invest as little effort as possible in your idea to validate it. The less you have invested, the less you have to throw away when it turns out the idea wasn't so great in the first place, and the easier it'll be to make that mental context switch.