Of course. It was called the greypages. https://en.wikipedia.org/wiki/Reverse_telephone_directory

In old movies you sometimes see people ask the operator to do a reverse search as well so I assume it wasn't a big deal.