And what if it fails to impress potential users? Then your web designer wasn't very competent, because that's exactly the sort of thing he's supposed to understand about your audience. A good web designer will bounce ideas off of focus groups and do usability testing, and then when the site goes live will conduct A/B tests.
Finally, the best way to make sure that the site doesn't impress potential users is to micromanage the web designer. You need to ask him or her about the web analytics numbers, but if you insist on wordsmithing on every page and monkeying about with fonts and layout yourself, you might as well just do the site yourself. (This is something I've seen repeatedly: the web site winds up being designed for and by senior management against the advice of the web designer, the potential users stay away in droves, and the web designer is blamed.)
Unfortunately, it's a lot like hiring a programmer in other ways -- the best designers will easily be ten times as good as the worst ones, and there's no necessary correlation between skill and rate.
So returning to the original question: look at the web designer's portfolio. Ask questions about web standards and (X)HTML compliance, usability, A/B testing, cross-platform CSS issues, readability, copywriting, audience. A good designer will understand these things and be able to explain them to you in terms you'll understand. Make sure you turn on your BS detector too, though, as a bad designer will try to baffle you with buzzwords.
And one of my favorite interview questions, when looking at a designer's portfolio, is "Why doesn't Site X validate?" The point is not to get the right answer, because there isn't one, but to figure out how the designer approaches these tradeoffs. I've also never interviewed a web designer who provided a portfolio of only valid HTML and CSS.