I think it is definitely a primary draw; I no longer use Vanced (used it years ago) but I remember the killer feature back then was being able to listen to audio from videos while having your phone screen off (a feature that YouTube eventually added to the official app).

I primarily use NewPipe. I enjoy the benefit of not having ads, but more than that, I enjoy the simplicity and the drastically lower system resource usage.