That's the case in traditionaly organised meetings, where I work we pull people in and kick them out pretty quickly. Meetings are 2-5 people max, everyone is contributing, and you get told you can leave if the topic pivots to something that doesn't apply to you.