I would think a bigger factor would be that the cables (IIRC) don't go in straight lines (which you did allude to).
(Since somebody else has already mentioned the pinboardiness of these comments... https://idlewords.com/2007/04/the_alameda_weehawken_burrito_... )
[0] https://www.nytimes.com/2014/04/14/opinion/krugman-three-exp...
[1] https://www.amsterdamtrader.com/2014/09/hft-in-my-backyard.h...
So you are still quite a bit removed from from the physical layers. Your packet will likely go through several electrical-to-optical and optical-to-electrical conversions, probably there will be some electric switches, plus multiplexers, all of which contain buffers. Then there is forward error correction in the physical layer which also requires buffers etc..
And you're obviously right that for many reasons the "straight path" might not be the path that is being taken, or even the fastest one.
Bottom line, estimating ping time from geographic distance is a very rough estimate. However, the longer the distance through an uninterrupted link (i.e. a submarine cable) the better your estimate, i.e. if you sit at in a google datacentre which is directly connected to their fibre backbone and do a ping to machine in a similar data centre on a different country you will get quite close numbers I imagine (I don't work for google). On the other hand if you sit somewhere in the mid-west at home and ping a server in e.g. NY or LA, not so much.
Ha, it only just struck me that could have been an NSA-type routing issue!?!
More than once I have seen a traceroute that takes you from Miami, down to Argentina and then back up to Cali.
1. Submarine cables can't go in a straight line, since they've got to, you know, go down to the bottom of the ocean. (Which, you may have heard, is quite deep.) Also, a cable with a length of several thousand miles tends to have some slack.
2. Your packets may take a very curvy route from one city to another, even when they're not geographically that distant. This may be because your ISP is bad (and has poor/limited routes), geographic or geopolitical concerns, or just because of the way the Internet infrastructure is built. On the US's west coast, I often experience latencies 60%+ slower than the theoretical minimum when accessing servers in the central or eastern US. (e.g. SF -> Des Moines, IA at 70ms).