I've been operating 64-bit kernels and user-space for at least a couple of years for embedded and network and server usage and not experienced anything to even remind me it's 64-bit rather than 32-bit.

I've used both the original Raspbian 64-bit builds and plain Debian 64-bit with the few required RaspberryPi specific tools added in. Kernels are all self-built (cross-compiled) keeping up with mainline Linux.