Absolutely, but only if there's a merge of lanes. If this is just the empty lane that people use to get to the front of the queue, like when there's a traffic jam to enter the highway and some cars are trying to cut the line, then you don't have to let them. I'm aware that this is not a common problem in normal countries like Germany where drivers generally behave well, but if you ever find yourself waiting in a line somewhere in Italy or say in Balkans, you'll often see cars trying to push past you by cutting the queue using empty and service lanes.