Well, we had good reason to believe that there were things with approximately their mass in approximately their area that emitted approximately 0 light to us. Which, yeah, isn't quite the same thing, cause you could imagine that if GR was slightly wrong, you could have something slightly different at the center than a True Singularity.
Didn't we know black holes 100% exist 20 yrs ago because of the orbits of the stars around the centre of the galaxy and before that because of quasars?