Since the hardware was extremely limited they could not afford to support both binary and decimal, so everything ended up being decimal. This was actually not an uncommon choice on some of the early computers. For example the ENIAC was decimal too, although that one wasn't designed for business.
There used to be a lot more experimentation in early computers before everything became standardized. The Russians even did some early ternary computers, there was a lot of variety in word length (36bit systems), the 8-bit byte wasn't even invented yet.
[0] https://en.wikipedia.org/wiki/Keypunch#/media/File:IBM26.jpg
I think the design was much more of a clean slate. Arithmetic is normally done in decimal by people so that was a reasonable way to start thinking about it.
I think this point is a good lesson for system design. So much design these days is driven by accumulated "wisdom" which is immediately buried with another layer of software, without those previous layers even really being considered as to whether they were appropriate or not. Those assumptions can often make the overall system more complex than it could have been.
Several incompatible IBM machines in the 7000 series also used arbitrary length decimal numbers, not binary words.
At Burroughs, the "Medium Systems" B2500-B4800 series were similarly decimal only with arbitrary length numbers. They got extended with fixed-length decimal accumulator and index registers, and competed with IBM's mid range 360 and 4331 systems. Building a decimal-addressed memory using binary-addressed chips got increasingly kludgy.
On the Bull Gamma 3: http://www.feb-patrimoine.com/english/gamma_3.htm
The Register had once a nice article on the occasion of the 1401's 50th anniversary: https://www.theregister.com/2009/11/17/ibm_1401_fiftieth_ann...
Kenn Shirriff has you covered on this, as well: http://www.righto.com/2015/10/qui-binary-arithmetic-how-1960...
What was the business justification for this? Was it ease-of-use for the businesses that leased these machines?