Looks like Nebula uses go-libp2p and all of the supported networks listed in the README use libp2p for their p2p networking. Mainline DHT doesn't support the same transport protocols that libp2p supports (such ash TCP+Yamux+Noise) which is probably why Nebula doesn't support Bittorrent