I generally wait with the traffic also when there is no bike lane/box. The only thing I ever worry about is getting hit from behind by some idiot not paying attention. I do try to mitigate this by turning my bike at an angle so that I look a little bigger.

This is all dependent on the traffic/intersection though, things vary depending on the circumstances.