It also is analogous to the fact that the sum of zero integers equals zero (a claim that, I guess, more laymen will find reasonable)
The analogy is a good one, as zero is the additive identity (https://en.wikipedia.org/wiki/Additive_identity) of the integers, and one is the multiplicative one (https://en.m.wikipedia.org/wiki/1#Mathematics)
Arithmetician (Ph.D. in number theory) here. This answer is completely correct.
As another reason why the factorization of 1 should be the empty set: suppose you have two positive integers m and n. Write S(m) and S(n) for their sets of prime divisors, counted with multiplicity.
Then S(mn) is the union of S(m) and S(n). We need S(1) to be the empty set to make this rule consistent.
Analogous to how log(1) is equal to 0.