Any encryption system has to exchange keys, and you can do it easily or securely, pick one. Failing the way you describe makes things cumbersome but does not compromise security.