Personally I think it'd be fine to e-mail back and say "sorry I don't have time to work on this with my current schedule, could I do an in-person interview instead?" That's me though, I would hope the rest of the industry is willing to work with people. Even just the response to the e-mail would tell me that they're competent enough to know their time management and that they have limited bandwidth. At some point you need to make a sacrifice of time though. Even if your hotel and plane ticket are being paid for to fly out to the place of business, you need to take that time off to go in for the interview.

I'd prefer to do a lot of that at home personally.