I looked it up and they date back to the 1960s on reel-to-reel tape recorders. The pause button was added later when companies found that they couldn't properly translate the word pause into every language. The idea is that the two bars look like the square stop button and look like a frame stopped while it is rolling to the next frame. The reasoning behind the square is that it's the play triangle without a direction; it is stopped in both directions.

This machine is apparently from 1961, so it must have been quite early in the 60s: http://www.reelprosoundguys.com/images/1962_GRUNDIG_TK_46.jp...