The other issue is that Mac OS X just isn't tuned for serving a lot of requests, Mac OS X Server is fine for calendaring and the other applications that come standard with it, and I would even suggest them just for ease of use, but for mainstream server applications you are better off with Linux/FreeBSD and for much cheaper as well hardware wise.
Regarding your experiences, could you be more specific as to what manual configuration might get broken by the GUI?
Also, regarding tuning, are you talking about app-level tuning (like Apache) or OS-level tuning? I'm curious because I'm a UNIX person without OS X knowledge.
Haven't used it in a couple of years now... might no longer be an issue.
The kernel design it uses (while having some advantages) is unfortunately slow at many tasks, which is detrimental to server performance. After years of tuning they've got it working well enough for the local user I suppose.
Not sure if it will ever be able to perform as quickly as a monolithic kernel such as linux.
"Not worth the cost" I can certainly see, but why is it a "terrible server OS"? Not saying you're wrong and I'm not in a position to argue the opposite, just curious.