I guess it depends on your definition of usability. Jakob Neilsen himself believes that user satisfaction is a part of usability:
http://www.useit.com/alertbox/20030825.html
I agree that it's possible to have eminently usable software that is ugly to a bunch of people. But it entirely depends on how that software is being used.
In the case of a web site, Jakob says it well (from the same article I linked above):
On the Web, usability is a necessary condition for survival. If a website is difficult to use, people leave. If the homepage fails to clearly state what a company offers and what users can do on the site, people leave. If users get lost on a website, they leave. If a website's information is hard to read or doesn't answer users' key questions, they leave.
I would add: if the site is ugly, people leave.
Your web site might be "eminently usable", but if people don't read it, it's a failure.
The implied irony was that a usability expert produce a site with poor usability, and it's just plain wrong. Visit the link. It's not even that ugly, frankly. It is spare, clean, and very well-organized. It just happens to look a lot like default HTML rendering. Shrug.