I'm a noob at this, but at a first glance it looks difficult to get all the peripherals and hardware acceleration working when compiling from sources. Anyway, I am really keen on getting one.
The hardware acceleration that is problematic is that of the Arm Mali GPU and of the video codecs.
This is normally provided in all Arm SoCs by binary blobs. Nevertheless, at least for the Mali GPU there is a reverse-engineered driver in the Linux kernel, which might be usable with RK3588.
In any case, this hardware acceleration is the same in all RK3588 boards, regardless of the vendor, it is not specific to NanoPC-T6.
The recompilation of the Linux kernel may be more difficult, because the right configuration file and modules must be selected before doing it, but it should be possible as most support for RK3588 is included in the mainstream kernel. Also U-Boot (the boot loader that loads the Linux kernel) should be recompilable from sources.
The hardware is more trustworthy than that of Intel or AMD computers, because it comes with the complete schematics and the technical reference for RK3588 is much more complete than for any Intel or AMD CPU.
A hardware backdoor could have been implemented only in the Ethernet interface of RK3588, but that is not used in NanoPC-T6, which uses Realtek Ethernet interfaces on PCIe lanes. Any hardware backdoor in those would have required a close and secret cooperation between a major Taiwanese company and a major mainland Chinese company, which is unbelievable.