Just because you cannot go to a miner's office doesn't mean that they aren't a middle man. The illusion of direct payments is simply untrue and you need to trust that there will never be more nodes from people that want to fraud the whole system than there are trustworthy ones. You trust complete strangers with your money that you can never meet and never complain about. Worse, you can't even sue them because they are most likely not in your jurisdiction and/or anonymous.