As choward already mentioned, I think NixOS is a very good choice: aarch64 is decently well supported with a bunch of people running NixOS on their Raspbys, and cross compilation support is really good too if needed. See https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi for more info, or hop into #nixos and/or #nixos-aarch64 on Freenode for help