It's worse than that. It's the friend saying "open any unlocked doors you want", then after you do it, he remembers he left some secret things in his unlocked bedroom and wants you arrested because he didn't know the lock was broken.

You are wrong about the law here.

It might be hard to prosecute, but just because I invite you over to my house I have absolutely not granted you permission to enter any room you want.

If you, for example, went into my office and started rifling through my file cabinet that would be a huge invasion of my privacy despite the fact that I (like many people) do not have a physical lock on my office or filing cabinet.