Considering the fact that peer review is done by volunteers, I think this a pretty weak argument. At best you could argue that the publishers are organizing the review process, but even that is suspect -- scientists organize themselves pretty well.
Peer review is orthogonal to copyright issues. Journal publishers serve almost no purpose in the modern world, and the copyrights they hold on scientific articles are doing more to prevent the dissemination of knowledge than to promote scientific research.
Now, once upon a time, journals served a secondary purpose, which was to distribute scientific results on a global scale. Back then, there was no Internet, and printing enough copies of a journal to satisfy the world's needs required industrial equipment. Fortunately those days are done and over with, but unfortunately we are still dealing with the relics of that bygone age in the form of academic publishing companies. Worse, in fact, since today's publishers are far greedier and for more profiteering than many of the publishers of the past.
Really, were it not for copyright, we could cut publishers out of the equation without any ill effects and have a net gain for society by removing all paywalls from scientific articles.
Open Access is a more subtle economic argument than I think many people here are admitting, and you have to recognise the value-added that journals provide. There are experiments out there (e.g. post-publication peer review, free-floating editorial boards) trying to challenge the journal model, but I'm not going to presuppose that they're going to be more successful than pay-for OA journals.
The paywall is arguably just for business though, a lot of papers get put up on sites like arxiv pre-publication anyway.
I don't see much in the argument from the view of the taxpayer though. In the UK a lot of public funding goes in to research and the papers are largely behind paywalls - Graphene research for example. Now, if we had government subsidized peer review journals - I could definitely get behind that.