I think context is important. If she is getting phone calls from guys asking for a date straight off then sure, I'd be annoyed too.

But if you develop a business relationship with someone it seems fine to ask if they'd like to go on a date - just like you might as something in the office.

From my observation, though, the former is probably more common than the latter ;)