I generally refer to them as agents or brokers. Those terms are more specific because, at least in the U.S., there is a legal distinction between the two.

I think the only times I've ever said "realtor" has been when it was informal. Having researched the NAR, MLS, etc. before selling my house, it seemed fairly obvious that realtor was a trade name and something I just assumed was trademarked.