My system, not their system, and since you're asking for a personal opinion here: immediately.

Well, you own the hardware, but not the software. So at least part of the system is not yours.

own the hardware but cant do what we want on it, because we're forced to use the software

Since we're discussing prescriptivism (how things should be) rather than descriptivism (how things are in a subset of legal jurisdictions): I didn't force them to leave a copy of their code on my device, but I own every single bit of electricity and magnetism and metal and plastic within it. I thus have the right to flip 0s into 1s, and 1s into 0s, or smash the whole thing. It's mine. Redistributing anything, admittedly, might be somewhat related, but would be a different discussion.

Any effort to disenfranchise me from those rights is, in my opinion, unethical. If a company were to try to do so under the guise of a semantic argument, I would find it even more distasteful.

Great question, which has now been answered by new laws like the digital markets act!

The "point" at which such things should apply is when they are large enough that anti-trust laws and pro competition laws apply to them. IE: when they are very very large and are worth hundreds of billions of dollars.