News for hackers. Stuff that matters.
Re:Apple suspends Siri response grading in response to priva... by misnome on Friday August 02, 2019 @08:25AM (#20592442)
> When people complained that putting microphones in your home 24/7 was creepy, the response was "that's nonsense, Siri is not sending anything unless you specifically ask for it".

This seems like a distorted representation, to support your argument, of what most the complaints and responses were. The gulf between 24/7 recording and "occasionally unintentional sending" is absolutely massive and you just merge them into "see, we were right".

The complaints were (and still are) conspiracy theories that they were streaming constant recordings and constantly listening; which they aren't; almost nobody is pretending this would be okay.

I've also not seen anyone who knows what they are talking about seriously claiming that /no person/ would ever listen to the audio samples - only that there isn't somebody _listening_ to your samples e.g. in a targeted, continual sense.

In fact the only part that does sound slightly worrying is that the evaluators get location information along with audio; this would probably be enough to reverse any ID anonymisation, and if the metadata is kept - requests by law enforcement, though I would hope the contractors couldn't control/request specific samples coming into them. In any case, Alexa is potentially worse than this because I can see a history and playback old recordings on my account, which means they are kept fully associated. Apple might do this with Siri internally, I don't know.

Re:Apple suspends Siri response grading in response to priva... by blub on Friday August 02, 2019 @09:37AM (#20592786)
You need to carefully read what you are quoting.

The microphones are in the houses 24/7, connected to power and the internet. That is creepy.

There were also several creepy situations involving these devices:

* Google home had a hardware flaw which actually made it record 24/7.

* all assistants activate by mistake when they think they recognize the activation word. Siriously.

* An Amazon Echo recorded a private conversation and sent it to a random contact of the owner

Re:Apple suspends Siri response grading in response to priva... by mikeash on Friday August 02, 2019 @09:52AM (#20592889)
Microphones connected to power and internet 24/7 have been in people’s houses since the 1990s (or earlier, if you consider the phone network to be equivalent). Yet it only became “creepy” when they started talking back. This tells me it’s an emotional response, not rational.
Re:Apple suspends Siri response grading in response to priva... by blub on Friday August 02, 2019 @10:10AM (#20593044)
Nobody considers the phone network to be equivalent :)

Even current smartphones do not turn on their microphones outside of a call, unless one has made the huge mistake to turn their assistants on, or there's an app recording - and at least on iOS that's controllable by permissions and marked clearly when the app is in the background.

Anyway, smartphones are creepy too.

Re:Apple suspends Siri response grading in response to priva... by mikeash on Friday August 02, 2019 @10:22AM (#20593148)
How do you know that phones don’t turn on their microphones outside of a call?
Re:Apple suspends Siri response grading in response to priva... by blub on Friday August 02, 2019 @11:46AM (#20593895)
This was answered in another comment: https://news.ycombinator.com/item?id=20593642

One cannot know with absolute certainty, only with reasonable certainty.

On the other hand, an assistant is listening all the time by design.

If one needs to know with absolute certainty, then they cannot use a mobile phone, nor probably a reasonably modern regular phone.

Re:Apple suspends Siri response grading in response to priva... by saagarjha on Friday August 02, 2019 @01:13PM (#20594651)
Modern smartphones have assistants that "listen all the time" in the same way that you're saying that smart speakers do (namely, they look for a wakeword).
Re:Apple suspends Siri response grading in response to priva... by Faark on Friday August 02, 2019 @04:06PM (#20596139)
No, modern smartphones allow you to have assistants that listen all the time. Disabling those is one of the first things I do, and I'd be pretty unhappy if it turns out those are still listening and wasting battery (even aside of privacy issues).
Re:Apple suspends Siri response grading in response to priva... by eridius on Friday August 02, 2019 @06:22PM (#20597121)
No the parent is right. They are not "listening all the time". They're essentially sleeping while a separate process handles the audio stream looking for the wakeword. When it finds the wakeword, it then "wakes up" the assistant which reprocesses the audio snippet to confirm the wakeword is really there (using more expensive analysis), before then actively listening to the rest of the audio.
Re:Apple suspends Siri response grading in response to priva... by blub on Sunday August 04, 2019 @03:35PM (#20608734)
The implementation is inconsequential as long as the smart assistants can be turned off.
Re:Apple suspends Siri response grading in response to priva... by blub on Sunday August 04, 2019 @02:43PM (#20608436)
Said smartphone assistants can be switched off and one still has a smartphone. Switching a smart speaker off turns it into a paperweight.
Re:Apple suspends Siri response grading in response to priva... by emiliobumachar on Friday August 02, 2019 @11:30AM (#20593754)
Back in the 1990s, their hardware design was simple enough that any mass surveillance would inevitably get exposed when the odd hacker took his device apart.

These days, I'm not sure at all. All we have is the law as a deterrent, and that's weak, especially to governments.

Re:Apple suspends Siri response grading in response to priva... by jerf on Friday August 02, 2019 @10:43AM (#20593342)
Cell phones, even now, can't be doing that much background processing unknownst to us, because they are connected to a battery, battery life is a big deal in a highly competitive industry, and we'd all notice if they drained very quickly.

ISTR there are documented instances of law enforcement and/or intelligence doing that sort of thing to phones, and the battery drain causing it to get noticed.

Hooking it up to an outlet is a legit different thing.

If battery tech or power reduction tech progresses sufficiently, including just by continuing on its current curve (most power reduction rather than battery tech), cell phones will rise to the same level of concerning.

(Cell phones of course leak location data full time, and some similar things like that. I'm not saying the current cell situation is awesome either. Only that there is a legit reason to be more concerned about gear hooked up to actual power lines than portable gear.)

Re:Apple suspends Siri response grading in response to priva... by tialaramex on Friday August 02, 2019 @11:29AM (#20593747)
Counterpoint:

Google's actual smartphone line (Pixel) ships with passive song recognition, default disabled. I enabled it because I was curious and it sips so little power that I left it on, but to function it has to be listening to the audio all the time so that it can say to itself "Hey, that is Sabotage by the Beastie Boys".

It doesn't use the network, or the screen (unless you power the screen on to find out what you're listening to) but it passively remembers every recognisable song it heard, essentially forever. So that means it is _listening_ it just doesn't tell anybody about it.

Re:Apple suspends Siri response grading in response to priva... by jerf on Friday August 02, 2019 @04:10PM (#20596170)
That's part of why I left my caveat in there about the future. We've had chips to detect wakeup words for a while now. From what I know about song detection, that's likely to be doing yet more stuff, but still nowhere near enough for full voice transcription yet. But it is only a matter of time before your phone can be doing full voice transcription, with good guesses as to who the speaker is, full time. Squirting a full day's transcript of your speech up would be very easy to hide in all the chatter a phone already has, and squirting up just selected "suspicious" speech could easily be hidden in just the metadata network traffic of a phone.

We have not historically been there, which informs people's about cell phones, and we are not quite there yet. 4 or 5 years maybe.

(Although I have some interesting thoughts on how to deal with that. It would be fun to take one of these "transcribing phones" into a movie theatre, then fire the MPAA at Google for copyright infringement.)

Re:Apple suspends Siri response grading in response to priva... by cameronbrown on Friday August 02, 2019 @10:53PM (#20598349)
Isn't a transcript transformative content and therefore fair use? There are still hundreds of variations for every piece of content.
Re:Apple suspends Siri response grading in response to priva... by jerf on Monday August 05, 2019 @10:17AM (#20614387)
I wouldn't think so. It's basically the script of the movie, minus the stage directions, and I don't think the MPAA is going to agree that's not copyrightable. There's no "transformation" there in the copyright sense that I can see, it's just a copy.
Re:Apple suspends Siri response grading in response to priva... by andromeduck on Friday August 02, 2019 @04:45PM (#20596494)
Doesn't your phone have to be awake a few seconds for that?
Re:Apple suspends Siri response grading in response to priva... by tialaramex on Saturday August 03, 2019 @03:57AM (#20599201)
What does "awake" mean here? If you mean do I need to explicitly wake it, then definitely no, to test this I just left it on a desk while I played Pulp's "Do you remember the first time?" and read a magazine, after a few minutes I'd read several pages and the song had stopped, I opened the phone and it told me I had listened to "Do you remember the first time?" by Pulp six minutes ago.

Does it in some sense "wake up" to do this? Yes, I assume so.

Re:Apple suspends Siri response grading in response to priva... by mikeash on Friday August 02, 2019 @10:52AM (#20593418)
I don’t think battery life is a problem. I think the examples you mention were from much older phones with limited local storage, so they were basically just in a call the whole time, and burning power operating the radio. A modern phone can save the data to local storage and upload it opportunistically.

We know that modern phones fan constantly listen to the microphone with minimal power impact, since that’s what these voice assistants are doing. That’s specialized hardware, but adding a little something to save the data wouldn’t be hard.

Even if we take it as a given that it can’t be done on batter power, phones get connected to outlets pretty regularly. Have you ever heard someone refuse to charge their phone in their house because it’s creepy?

Re:Apple suspends Siri response grading in response to priva... by 6gvONxR4sf7o on Friday August 02, 2019 @11:02AM (#20593482)
It's totally rational. When the mic activates only when I physically press a button or pick up a device and dial a number, it's different from it activating itself. I'm in control of the former, and am happy to have it in my home.
Re:Apple suspends Siri response grading in response to priva... by mikeash on Friday August 02, 2019 @11:05AM (#20593504)
How do you know you’re in control?
Re:Apple suspends Siri response grading in response to priva... by 6gvONxR4sf7o on Friday August 02, 2019 @11:19AM (#20593642)
I don't. It's a spectrum.

I don't know that an old-school land line phone isn't always listening. But it was made by some disinterested party who just makes hardware. It was a simple logical device designed to rely on a simple mechanical switch. It could be listening, but it's understandable by someone like me and there's no incentive for anyone involved. I don't really know if I'm in control, but it's reasonable to think I am.

I don't know that a modern voice assistant isn't always listening either. It is an incredibly complex device usually designed by an entity whose business makes hundreds of millions of dollars based off of the inspection of the personal data of people like me. It is a complex full functioning computer. It uses a statistical algorithm that I know will incorrectly activate sometimes. I cannot understand or predict when it will listen. I cannot understand or predict when it will send a recording somewhere. It receives updates frequently. Because it's a full computer with an internet connection, those updates could make it do a huge variety of things. I know I'm not in control.

Re:Apple suspends Siri response grading in response to priva... by mikeash on Friday August 02, 2019 @11:29AM (#20593751)
This seems reasonable, but a cell phone would be close to a voice assistant on this spectrum, or even in the same place.
Re:Apple suspends Siri response grading in response to priva... by 6gvONxR4sf7o on Friday August 02, 2019 @12:10PM (#20594124)
Yup. E.g. Siri
Re:Apple suspends Siri response grading in response to priva... by lancesells on Friday August 02, 2019 @09:51AM (#20592879)
For some reason smart speakers give me pause but not a smartphone. A smartphone is kind of the same thing (microphones listening) when plugged in with the addition of a camera and a host of other sensors. I guess because a phone's main function is to not listen to us that makes the speakers much more worrisome to everyone?

Even something like an Apple Watch is the same thing with the addition of reading your vitals.

Re:Apple suspends Siri response grading in response to priva... by fastball on Friday August 02, 2019 @10:09AM (#20593032)
I mean, the "main" function of a phone IS to listen to you. For phone calls.
Re:Apple suspends Siri response grading in response to priva... by blub on Friday August 02, 2019 @10:13AM (#20593064)
The main function of a phone is to make phone calls. In order to do that, it activates a microphone during the call.

This is not the same as listening, never mind always listening. A home assistant must always listen in order to recognize the activation words.

Re:Apple suspends Siri response grading in response to priva... by fastball on Saturday August 03, 2019 @01:13AM (#20598804)
If you don't trust home assistant vendors to not randomly send non-queries to the cloud, I'm not sure why you'd trust phone vendors to not have the microphone on when you haven't given it permission.
Re:Apple suspends Siri response grading in response to priva... by blub on Sunday August 04, 2019 @02:48PM (#20608465)
The latter is more unlikely compared to the former, which is by design. Furthermore, at least on iOS Siri can be completely turned off through a configuration profile.

This is a reasonable approach for someone whose thread model allows them to use a smartphone...

Re:Apple suspends Siri response grading in response to priva... by pdkl95 on Friday August 02, 2019 @02:27PM (#20595358)
> The complaints were (and still are) conspiracy theories

The people making those complaints are probably not crazy; it might be a good idea to try to understand their actual concerns instead of jumping to this inaccurate (and insulting) misrepresentation.

> they were streaming constant recordings

That would be a waste of bandwidth. Obviously a constant stream isn't happening. While a trivial noise gate and high-efficiency encoding could easily[1] allow almost constant recording of voice for trivial amounts of bandwidth, that isn't really the concern.

> constantly listening

It has to be constantly listening, or it wouldn't hear the wake word. However, this too isn't really the concern.

> in a targeted, continual sense

Dan Geer, on "targeting"[2]:

>> There is no mechanistic difference whatsoever between personalization and targeting save for the intent of the analyst. To believe otherwise is to believe in the Tooth Fairy. To not care is to abandon your duty.

Targeted surveillance is by definition rare and often hard to avoid. I'm far more concerned about non-targeted, sparsely-gathered data that surveillance capitalists like Google/FB/etc can use to refine their model of your pattern-of-life[3]. While this touches on a few important issues, yet again this isn't why I consider always-on, remotely-controlled microphones to be a serious problem.

The big problem is that you're normalizing the idea that remote 3rd parties might be recording in areas traditionally considered private. Kyllo v United States[4] created a bright-line rule that asks if a technology (in general, not any specific product) is considered to be "in general public use"[5]. If it's not, using that technology is considered a "search" for 4th Amendment purposes, and a warrant is required if the police want to use it. When a technology is "in general public use"[5], the police no longer need a warrant to use their own voice-activated internet-controlled microphones to see the "details of a private home that would previously have been unknowable without physical intrusion"[6].

I don't care what these devices may or may not be recording right now. I do care that normalizing of the idea that voice might be recorded inside the home by a remotely-managed device is eroding my future 4th Amendment rights even if I never buy an Alexa/etc.

[1] https://news.ycombinator.com/item?id=16346416

[2] http://geer.tinho.net/geer.source.27iv17.txt

[3] https://en.wikipedia.org/wiki/Pattern-of-life_analysis

[4] https://caselaw.findlaw.com/us-supreme-court/533/27.html

[5] used multiple times in ruling[4] (esp. section II of Justice Stevens' dissent)

[6] see [4], 2nd paragraph

Re:Apple suspends Siri response grading in response to priva... by jtbayly on Friday August 02, 2019 @09:36AM (#20592776)
Many apps turn the microphone on and leave it on. Even if they aren’t recording it all or having somebody listen, it is clearly to spy on you to see what you are doing. (Ie did you hear some ad?)

That’s not a conspiracy theory, and it is practically indistinguishable from constant listening for the normal person.

Re:Apple suspends Siri response grading in response to priva... by tinus_hn on Monday August 05, 2019 @05:29AM (#20612614)
If an app in the background has the microphone on on iOS the OS will show a red indicator warning you. Also apps can’t access the microphone without your permission.
Re:Apple suspends Siri response grading in response to priva... by freehunter on Friday August 02, 2019 @11:22AM (#20593664)
>Many apps turn the microphone on and leave it on.

Source?