That's just not how keys work. If someone finds the key to my house and uses it to open the door and take something, it's still theft.

Except most legal systems don’t see it that way. When you say this, they hear “if you have a copy of the key to a house/car/etc, then you also own it”. Cryptocurrency theft is a crime by the basic definitions of the legal system.

That is not how the law works. It's equivalent to knowing your bank account number and routing number and making an unauthorized transaction.