Crosstool-ng is nice, but if you have even a mildly out of the ordinary arch/libc/kernel version/etc combination it is still a massive pain in the ass. Typically I'd rather build the kernel on device overnight instead of spend manhours getting crosstool-ng to work as I want so that I can build on beefy hardware.

It tends to only be worth it if you are going to be using it more than a few times.