> Is skipping the context switch enough to make a meaningful performance different?

Only in (very) high-bandwidth cases and (maybe!) for low-power cases. For most things it's just added complexity for no benefit.