Unfortunately, yes, but you don't necessarily have to use the TrueCaller app on Android/iOS, the website works just fine. Besides, TrueCaller and WhatsApp won't be the only apps to steal your phone book, and with 100M and 2B installs respectively, the jury is already out on their utility.

You can use Whatsapp without giving it access to your phonebook. It complains initially but it works fine.