Exactly. Electronic arithmetic hardware predates WWII. IBM had an electronic multiplier working. ENIAC was a giant plugboard machine. It's not that people didn't think of stored-program computers before Turing. It's that there was nothing in which to store the program.
IBM built machines with plugboard memory. Relay memory. Electromechanical memory. Punched-card memory. Punched tape memory. Drum memory. Look at the history of the IBM 600 series machines, a long battle to get work done cost-effectively with very limited memory.
Delay line memory was sequential and slow. Willams tubes were insanely expensive per bit. Core memory was a million dollars a megabyte until the early 1970s and didn't get much cheaper. There was plated wire memory, thin film memory, and various ways to build manually updated ROMs. All expensive.
Then came semiconductor IC memory (1024 bits in one package!) and things started to move.
This is way out of my knowledge domain, so I'm curious why mercury delay loops were used in the first place.
Thus, there is an inherent tension between making the tubes longer (more storage per tube) versus keeping them short (lower access times). This tension is inherent to all forms of "delay" memory, including the wire torsion memory grandparent mentions.
I can't find the reference now, but I remember reading about some early computer engineers (possibly on Whirlind?) contemplating the use of a microwave retransmitting station as a form of memory, essentially creating a "mercury" delay line with microwaves in the atmosphere.
This is how desperate people were for anything even remotely affordable which could be tortured to behave somewhat like memory! No wonder Intel made a bundle when they started offering chip memory. The stuff it was replacing was just totally inadequate for the purposes many people wanted to put it to.