I have heard that argument made, but it seems to be a peculiar result of the government classifying private key encryption as a weapon to keep it secret during the cold war.

I think that the founding fathers would have believed people had a right to encryption, but that it was covered by the tenth amendment or first amendment. I believe that at least some of the founding fathers did use codes and invisible inks. I can't imagine any of them would have seen codes as weapons.