I do similar but using wireless headphones, to get on with other tasks without being so affected, or affecting other people.

Even then, I find myself semi-paralysed by the anticipation. It's hard to focus on something else, in case I don't notice the pick up and then they hang up. A callback would be much better.