They estimate spending just over a million dollars this year, funded via members (university libraries contributing a few thousand dollars each) and grants.
However, as I always point out, we must remember that the costs of a published journal are higher for reasons other than profit: journal articles are typically converted to HTML and XML via semi-automated processes with manual proofing and tweaking, production staff and some editors (for the biggest journals) are paid salaries, peer review involves chasing down deadline-abusing academics over weeks or months and building some kind of review tracking system, articles have to be reliably archived (e.g. lockss.org), and so on.
Then add that most journals are still printed and distributed by mail for some reason, and you have extra full-time staff and equipment.
I wouldn't want to throw all of that away and just keep the arXiv. Ditching print makes sense, some things can be better automated (we need better tools than LaTeX or Word to prepare papers, so they can easily be produced in PDF, HTML, or semantic XML form), and some services are unnecessary (I've had journals copy-edit my articles and make no useful changes), but I don't want to just read crappily-formatted base LaTeX from the arXiv with no frills either.
(Incidentally, PLOS is non-profit and still has to charge over $1000 per article in PLOS ONE, to pay for the professional staff and all the manual manuscript conversion gunk. Biologists love Word, so converting must be hell.)
What would you suggest as an alternative to LaTeX or Word?
I'm just curious; I use LaTeX regularly and think it works just fine.
I don't know of any good alternatives. LaTeX succeeded because of its excellent extensibility, so the core can be used to produce almost anything. I'd like to find similarly extensible tools that can cleanly produce XML as well as PDF. Maybe Pollen (http://docs.racket-lang.org/pollen/) will be one.
There's also Scholarly Markdown (http://scholarlymarkdown.com/), but I suspect Markdown's lack of easy extensibility will doom it, since you'd have to write documents that fit narrowly into the features they provide. (What if you want a "remark" environment and they don't provide it?) reStructuredText is also an option, since it's built for extension, but it's not very well-known outside of Python circles.