It's the transportation "mode shifting" difficulty. Per the AI, the term of art is "Pure Transfer Penalty". It's the "ick" when doing bike => bus => bike instead of "only bike" or "only car".

Mode switching has a cost. Usually std::sort is good enough compared to picking the prime optimal algorithm for your expected shape. Just call the function and get on with your day.