There are no internet authorities that control mail delivery, either, BTW. Many sites use heuristics to control mail acceptance, however, including third-party whitelists and blacklists. It can suck if you get on a blacklist, but experienced mail administrators only use whitelists and blacklists as one of many metrics regarding mail spamminess. Indicators of good mail like "most of the words in this message are non-spam words" often provide an order of magnitude more ham points than blacklists contribute spam points, so important messages will probably not be dropped even if you're on a blacklist. Of course, many sites have less clever schemes because there is a lot of spam and not a lot of sysadmin time to waste tweaking spam filtering rules.
Mail is hard.
In any case, from a decade of listening to the random things he's had to deal with, I certainly have to agree with your final comment.
I think there are probably some MTAs that will STARTTLS for normal SMTP connections, but again, I've never heard of anyone using SSL/TLS as a spam-filtering criterion. (I might look through my old logs to see if anyone other than me ever issued STARTTLS on my mail server. But I'm guessing the number was near zero.)
Last year I moved and my IP address changed (ISP didn't). I expected to lose the "reputation" the previous one had built up. But I did not have any problems whatsoever. I do hear rumors like yours every now and again but I have a hard time believing them.
I think that large email providers, the likes of Gmail/Hotmail, actually look at the email addresses / domains / servers that their clients send email to. Then they assign trust to those tokens. If you're a large provider, you can do many things with the data you get from your own customer's behaviour. How about looking at accounts that have been in use for some time, seen regular web interface action, and send email to other @hotmail/gmail accounts that actually get read and not flagged spam? If those accounts send mail to my mailserver, then my mail server / domain must have something good going for it. Well that's what I would do if I were running a huge setup anyway...
As for incoming spam: I'm using various postfix tricks, greylisting, and dspam. I have no problems. I should write a howto ;-)