It's happened to me though; one scenario is that you created a couple of merge requests for the same repo over the course of a day, and the next day feedback from code review or testing comes in. You end up having to switch to various branches to fix the reviews.
But that's really not that big of a deal. The context switch is worse than the git checkout call.
Sure, but its not like the different changes just don’t exist if you use only master, branches are a way of managing them separately and only merging when you are ready, if you don’t use branches, you have to have some other way to manage this. If you don’t, and its not a problem, then you could also just not work on three branches at once.
All build systems (that people actually use) are really bad.