That song has had many covers but my favorite is a local band, The Woolies, that had a Billboard top 100 hit with it in 1966:
So obviously what's "proper" or not depends on what dialect you're trying to conform to. I can believe that if you're part of that narrow, elite sliver that would naturally use 'whom do you trust', the "incorrect" use of "who" as the object of a sentence must seem garish, but to the rest of us, trust me, "whom do you trust" sounds ridiculous.
Live and let live?
(1) "Whom do you trust?" http://www.economist.com/blogs/johnson/2012/04/grammar
In both "to whom is the message addressed" and "whom do you trust", "whom" is an object, not a subject.
All that being said, it is increasingly a common and acceptable practice to use "who" as both an object and a subject, so it's probably not worth losing too much sleep over.
Who made this? He made this. Whom do you trust? I trust him.
This mnemonic is predicated on answering in complete proper sentences. I realize that in current colloquial English, "Who made this? Him." is certainly an exchange one might encounter.