According to the Oxford English Dictionary, the expression "mylke of almaundes", i.e. almond milk, was used in 1425. So I'd say the use of "milk" to refer to non-dairy products is fairly well established.