You don't need a middle man for physical cash (let's use USD as the example), but services like credit cards and debit cards are extremely convenient. Middle men can certainly be of great value to the bitcoin economy, by providing similar services to the ones that already exist on top of government currencies.