Yes, you're right. The size of the stack is relevant to how much memory each switched-out task takes up, but not directly to how costly the switching process is.