I think the thing that is concerning to me is not so much that users don't care about the envelope from, so much as it is that other email providers' anti-spam measures may block my email if some spammer start spoofing me. Then, poof! I can't email any gmail accounts anymore.