Yes, any app that uses Google Maps is backdoored, but a secure messaging app needs to be held to a higher security standard than any old app that uses Google Maps.

It is still possible to run a phone with Signal without any proprietary google code on your phone (see: microG).