Personally, I wouldn't put it past either company to massively violate my trust if the incentive stars aligned just right, and I think it's silly to pretend otherwise.
The problems of putting a surveillance device in your home and carrying one in your pocket are roughly equivalent, and the rest is a debate about which huge collection of people with a wide array of beliefs and ethics is gooder
A more likely path is law enforcement's habit of putting them in the PR hell of "we could have saved Little Tiny Baby X if these companies weren't so hell bent on protecting perverts and terrorists". Once you have the device, it's only a matter of someone making a sympathetic enough public case that they need access to it.
On that front, I think Apple is taking more proactive steps to keep themselves out of the situation, I'm just not sure it's possible to avoid it completely.