At it's core, libertarianism makes no statement about outcomes, but simply demands that people be free to make their own choices without being subject to force or fraud, and are then responsible for the outcome of their actions. Nobody is making any Pollyanna'ish guarantee that such a system will result in the most possible desirable outcome for everybody involved. The claim is that it's just the right thing to do, that most people will be better off (economically) than under other systems, and that you accept that every system - this one included - has its pathologies. You can't get much less naive than that.
There it is, that is the idealistic part.
But, again, none of this is really the point. Every system is flawed in its own ways, and the libertarian approach simply acknowledges that, rather than trying to distort reality and pretend that a "perfect" system is even theoretically possible.
What libertarianism is about has been a subject of debate between libertarians for decades. There are others who see it as an idealistic belief.