> The researchers then decided to see if they could use data collected on mute from that app to infer the types of activities taking place in the background. Using machine learning algorithms, they trained an activity classifier using audio from YouTube videos representing six common background activities, including cooking and eating, playing music, typing and cleaning. Applying the classifier to the type of telemetry packets the app was sending, the team could identify the background activity with an average of 82% accuracy.
How is this not extremely concerning for anybody who cares about privacy?
How about we not make the default that companies can do whatever they want and users have to take steps like a hardware-muted mic (which isn't always an option) to ensure a basic expectation of privacy?
The way I read that is, only one of the apps actually sends audio data to the server when the mic is muted. I'm not sure why they don't say which one, and I'm not sure what is meant by "occasionally gather raw audio data" but it could be as innocuous as the mute button not updating and a half second of audio being sent before muting starts. Nobody is building a machine learning profile out of that.
The real story here should be that one app where the mute button doesn't actually work. The others are all operating normally as far as I can tell.
Yeah, that's very annoying.
Zoom, for example, will tell you, when you are muted and you begin to speak. It's a very nice thing.
I have a microphone (that wasn't expensive) that has a hardware mute. I use it when I really want to make sure I'm not heard, even for "speaking" detection.
MS Teams has that feature too.
TBH I'm not surprised: when you mute the mic in an app, you're still letting the app in control. If you really want to be safe, you need to mute at OS level or in hardware. That's why cameras have a hardware cover in modern laptops.
> How about we not make the default that companies can do whatever they want and users have to take steps like a hardware-muted mic (which isn't always an option) to ensure a basic expectation of privacy?
Sounds nice on the surface, but is ultimately silly. Sure, it'd be nice if everyone did the right thing, but you can never guarantee that, and hence if you really care you need to perform the mute at a lower level than what the app has access to.
And the increasingly (or is it just perceived?) shady behavior of big companies are not helping.
I am afraid this eludes our leaders across the board.
It would be so nice to have a mute-mic button which lights up when muted.
Most of the actions just duplicated existing UI elements.
So when you go to the deck to perform an action, you mentally context switch away from the keyboard, so your brain is looking for different clues.
So even with a touch bar + function keys, the function keys stay in keyboard context, but the touch bar requires that switch in any case. Maybe taking your eyes to the touch bar is enough of a context switch, mentally.
The problem was that it was too much of a tradeoff. Either a touch bar or function keys. I need function keys. The touch bar was useless for this purpose (no tactile feel).
Apple could easily have done both. There's more than enough space even on the 13" macbooks. It would have taken a bit of space from the vertical range of the touchpad but it's already comically large anyway.
I carefully made sure I didn't mention the touch bar. I waited until 2022 to finally be able to buy a MBP without a touchbar (and with magsafe charger).
I would very much like a physical button with a reassuring tactile feeling to it. Like the mute button on the function keys row.
The mic mute button of course works at the OS level but still...
Zoom was discussed 2 months ago in "Why is the Zoom app listening on my microphone when not in a meeting?"
For libel and slander cases though, telling the truth is a valid defense.
Which is neither here, nor there, as some are more likely to sue you than others.
>For libel and slander cases though, telling the truth is a valid defense.
Which is neither here, nor there, again. Not everybody wants the hassle of going through a lawsuit or the time and money costs associated, even if they have a "valid defense".
Should i hazard a guess, it would be either Zoom or Teams.
As well as, I assume, to archive a text indexable/searchable log of your conversations. I say this latter part based on multiple experiences I've had receiving clearly targeted advertising for topics I make random passing jokes or commentary about in live conversation, but which I've never once searched, click-imprinted, etc. for online. But, this is purely suspicious speculation. The live closed-captioning feature is a real thing and a thing I could absolutely see resulting in Google always sending audio streams to their backend for.
It makes some sense here on HN where people often post under their real names and want to think carefully before badmouthing a former employer, or otherwise picking a fight.
I agree that journalists have little excuse.
Except wanting anyone, ever to talk to them again about something controversial.
Bob Woodward has gotten PLENTY of high level politicians to talk to him candidly even though he's publishing what they are saying often in an unflattering light.
If someone as famous as the watergate guy can STILL get politicians to talk to him, what does a random journalist have to fear?
I mean, FFS, the gamer nexus guy got newegg to talk to him even after blasting them about ripping him off and letting them know "we are recording everything".
Journalists are acting like a single inkling of a bad word said will lock them out of access to everyone everywhere. The truth is, there are so many journalists out there that one could make a career of asking hard questions and publishing unflattering statements and the STILL would likely not be recognized by most individuals if they asked for an interview.
Woodward is absolutely one of the few exceptions, if not the only one.
He publishes on a long timescale with dozens of sources, not a short timescale with one or two.
So everyone knows the story will come out but not immediately, and everyone knows everyone talks to Bob.
But like it or not, most (not all) of the people he talks to have less to lose, because they are at or close to the apex of power in DC; there will always be jobs for them elsewhere in the USA, in a thinktank or on a board somewhere.
(It's also worth noting that Woodward's most famous source was anonymous for essentially his entire life)
In the real world outside seats of government, talking to a journalist on the record about stuff you should not often means you are the single source -- the only person who made the unflattering or damaging story possible, in a story that maybe won't wait even a week to be published.
It puts you out there on your own, gets you fired, makes it difficult to get immediately re-hired, and I suspect for an American with a family in an at-will state makes telling the truth on the record a luxury they can't afford.
Nothing about that situation has bearing on whether people first think of safely concealed sources or being publicly dragged through the mud like Chelsea Manning and Edward Snowden before sharing things far more important than entitled rants about customer service for entertainment-related products. My lord.
Why not finishing their research first, discussing with lawyers how not get sued, and contacting the company about it so that they can fix it or comment on it, and only then publishing the proper, responsible research that is actually useful to someone?
People can be irrationally attached to the things they use.
As a farm kid growing up, you didn't step between two guys arguing the merits of I-H and J-D farm equipment (1970s...)
It is doing this for the ultrasonic room detection feature, which can be turned off.
You're reading it incorrectly.
The paper outlines that as far as they can tell Webex is not sending audio when the mute button is pressed, but is gain and other parameters, based on this audio. And this has been reported to Webex, and they are investigating.
There should be a line between "companies doing whatever they want" because of some implied "nefarious" reasons, and "companies doing whatever they want" because their customers want a better experience even if it has security/privacy implications.
So sure, Teams gets some things right, UX-wise, but for every "notify the user if they're speaking while they are muted" there's a "randomly scroll the chat back down while you're looking through history", a "randomly make it impossible to erase a link with backspace in the chat box" or a "don't let people see who is participating in the conference", and there's little incentive for them to avoid crap like it and avoiding whole categories of bugs because people are buying it for other reasons.
For the vast majority of users, price beats UX. If a company can keep their app free by selling user data, they will out-compete paid alternatives, regardless of the UX.
Is Zoom audited? Zoom had been lying for about having end-to-end encryption, for example, until they were caught by the US Federal Trade Commission. Surely, something like that would have been discovered earlier in an audit, if they were audited and the audits were worth something.
They were also sending data to third parties like Facebook and Google through their SDKs.
https://arstechnica.com/tech-policy/2020/11/zoom-lied-to-use...
https://arstechnica.com/tech-policy/2021/08/zoom-to-pay-85m-...
There are so many alternatives, how is it Zoom has any business at all?
Zoom was the first videoconferencing software I experienced where the first 15 minutes of the meeting was not spent with "can you see me," "can you hear me," some people falling back to dialing in to a speakerphone, and one or more out-of-band calls to various participants to troubleshoot problems.
Zoom was click a link. And it worked. Nobody cared much about anything else beyond that.
There are only two videoconferencing platforms I've never had any problems with: Zoom and Google Meet. I don't trust either company, but sometimes you just have to get your work done.
I often call it "golf-course-ware" the sales person goes golfing with the executive, they discuss features and prices and discounts ober the match and the executive typically doesn't have to use the software but only their employees or the assistant.
Interestingly Zoom for me was a game changer in usability and it spread during pandemic, when executives where at home partially without their physical conference room with video conf setup and without assistant.
Anecdotally, privacy/security seem to be on the bottom of the stack. Platform support and necessity for work are above them all.
1. When the mic is soft muted, information is getting sent to the conference provider which could leak information about private matters.
2. When the mic is not muted all information is definitely going to the conference provider in a way they can decrypt so they can mix it.
That is to say that when using most conference software, you have already granted then access to contents of the meeting.
If you can't trust them to not miss use information they get when the mic is off, then you also can't trust them when the mic is on.
https://mashable.com/article/signal-end-to-end-encrypted-gro...
Jitsi has supposed e2ee videoconferencing (on Chrome - they used some chrome-specific API for processing) for (I believe) at least a year or two.
Element has native E2EE for 1-to-1 calls, but uses Jitsi for group calls. Native E2EE group calls actually also exist, called Element Call (https://element.io/blog/introducing-native-matrix-voip-with-...) but they're yet to be integrated into Element and specced into Matrix, I believe.
I don't believe this could be true, and the linked article doesn't have the word "mix" anywhere. I imagine that there is no mixing happening until after decryption on the client device. Of course this means that every audio source goes to each recipient discretely, which means more bandwidth, but audio (especially near-silent moments therein) is lightweight enough for reasonably sized groups. Obviously this same n^2 scaling issue happens with the video anyway which is never mixed.
More info is available at my blog post about it: https://signal.org/blog/how-to-build-encrypted-group-calls/
I manage some properties for a family member on the side, one of which is in a very bad neighborhood. When I travel to this neighborhood I have a certain state of alertness that I would not normally have in my boring suburbistani neighborhood. This is better known as "situational awareness" - the man approaching me in my own neighborhood is likely a just having a friendly conversation, the man approaching me in bad neighborhood is guaranteed going to at least try to bum a smoke off me, which I don't have as I don't smoke, and will likely act belligerent if I refuse to give him money as a follow-on to the request for a smoke.
Contextually, I expect a video conferencing software to be listening to and watching me even if it doesn't necessarily reflect in the UI, it has the capability and is actively meant to do so. As such, I explicitly don't have any form of sensitive conversation in the vicinity regardless of status. On the other hand, I do not expect it to do so when not running nor my laptop to do anything similar.
Perhaps there is a legitimate criticism to be made here of poor UX around "not listening" - but to paint this as an "extremely concerning" issue is sky-is-falling critique. This over-the-top concern seems further alarmist in that both my laptop and phone display clear and obvious warnings to the user when the microphone is hot.
But that's true for literally all applications running on your computer. Evil software running on your machine can do all sorts of bad things.
The problem, as reported in the article, is that apps are not making use of the OS mute, but are instead still reading from the microphone, and some are even passing the readout to their servers.
This is why I personally insist on using the web version of streaming software over an installed binary.
Because they're not actually doing that?
These researchers did everything they could think of to come up with the most concerning headline.
I imagine someone, somewhere is going to make a video conferencing app that closes the audio interface every time you press mute. I also expect few people will use that option because it adds additional latency every time you unmute.
I want my mute button to work ASAP and I don't believe Zoom (or anyone else) is interested in whether or not I'm eating while muted.
>> Applying the classifier to the type of telemetry packets the app was sending
Are you sure they’re not? The used these algorithms on telemetry packets sent from browsers. I see no reason to give companies whose revenue is built on ads the benefit of the doubt here.
Besides - is it really super valuable to know that while I (thought I) was muted that I was cooking? Zoom's going to find out I... eat?
If they are, the continuing video feed is pretty likely to answer their question.
But the detectives with a search warrant are pleased to be able to listen to your private conversations.
The secret police who do not need a warrant (or legality) are glad to be able to listen too.
The staff at Zoom are happy to spy on you, probably, for a small reward.
A proper mute removes those concerns. A mute that does not mute is inviting a lawsuit
The technical cost of deploying this is probably large, and the cost to reputation immense if they were caught doing this. By comparison, giving people the additional sense of privacy by actually turning off and on the mic is likely more than outweighed by the annoyance of lag between turning on and off your mic and being heard by the other chat members.
Although they could do something like write random bits of audio to the stream when the mic is muted in software. That'd at least let users know that the actual audio isn't leaving their device. But the hardware peripheral activation is probably not going to go away.
You're implying government policy for how companies operate in this area... Which is worth pursuing, but we all know that usually ends up half way effective, requires a cat and mouse game of auditing and enforcement, or big companies playing fight club math with the fines.
Even if this was already the case for this particular issue, as users we end up never really being sure if a company is violating that particular requirement.
> users have to take steps like a hardware-muted mic (which isn't always an option) to ensure a basic expectation of privacy?
This should be the default, just like how operating systems and networking evolved over the past 30 years starting with a "trusts everyone" attitude towards a "trust no one" by default. We need to assume most companies are potential bad actors, the hardware and software that comprises the basic operation of our device needs to provide the user with facilities to control flow of information separate from third parties, especially when it comes to input devices. In the case of microphones and cameras hardware switches should be the norm, or at minimum indicator lights.
This could also quite easily be a government policy for hardware vendors, and I suspect it would be more effective... It only has to reach a threshold after which users expectations shift to force manufacturers hands, so it's direct effect need not be as comprehensive to be effective.
There are also laptops with hardware microphone switches (eg. https://puri.sm/products/librem-14/ or https://frame.work/).
I am muting because I don't want the sound in my room broadcasted... if it was silent, I wouldn't have to mute!
Not to mention push-to-talk has solved this issue for almost a few decades now.
Because
1) most group calls that need people to be on mute most of the time are useless, boring, snooze fests, most attendees don't care about, so those 'professionals', who are caffeinated zombies half asleep, will space out and forget the status of their mic within 10 seconds of toggling it
and
2) most chat apps suck at drawing attention to the status of the mic and, if you have multiple monitors, you can be staring at one monitor (Jira, reddit, Redmine, HN, VS Code, etc.) while the chat app and the status of the mic is being displayed on another monitor where you're not looking
It's a mistake super easy to make. Still, better be safe and make the mistake of being muted all the time, than forgetting to mute yourself and have participants hear something you didn't want them to hear.
Ideally I'd want a feature that gives the image on all my monitors a nuclear red vignette, or something like that, whenever my mic is hot, so I don't have to keep paranoidly glancing at the mute toggle every couple of minutes, to make sure my mic is still muted, so they can't hear me mumbling on how incompetent management is and on how useless this meeting is.
I know people that literally retired early when they were forced to use PCs in the office. Over 30 years later, many people can barely use the most basic features of their computer. All to say, I’m not surprised this is an issue and I don’t see people as a whole digging their way out any time soon.
Some people don’t intuitively track the state of the video conferencing microphone, especially if they have cognitively involved jobs or lots of distractions. Mine are 1) the inability to resist that little self esteem boost from disdainfully highlighting inanity of other people’s shortcomings, and 2) making snide comments.
They’re both super obnoxious but I’m working on them.
Still, PTT is the solution, preferrably in hardware. Not supported in sw anyway by e.g. Teams. In hw it keeps the mike-on symbol lit, and the device powered. Always having to push prevents ever forgetting to do so.
Discord's input handler sucks, uses semantic keys, not keycodes. Can't be mapped to an otherwise disabled capslock. TS and mumble can do that. Compared with those Teams audio looks like a toy.
Personally I just have a headset with hardware mute functionality and a big red circle showing me it's muted. It remembers its muted status, so I just mute it by default and default my OS to use the headset's mic. That way I know quickly and easily when I'm muted and when I'm not, though even then I have small mistakes in the mornings when I'm tired. Over time I've optimized my meeting workflow because my company has gone all remote and I'm in a lot of meetings.
It also can keep the Teams mute status in sync as long as I don't touch it in the app myself (I believe through Teams detecting whether the mic interface is marked as muted or not, so it isn't exclusive to my mic).
"If you don't want to install our app, you can just install our browser."
For me, with this method, text chat works a bit (often forgets chat history on reload). Notifications get dropped all the time. No video conferencing at all. Sometimes I have to discuss with colleagues, why my teams acts weird.
As seems to be pretty common, for the sake of privacy we do stop sending audio to the media server. That's a tradeoff, since we're still susceptible to losing a little bit while the audio connection resumes.
Edit: as others have mentioned, also useful to keep bluetooth headsets in two-way audio mode rather than reverting to audio output mode, since that's really disruptive.
The first time I had ever heard of Zoom, it was long before the pandemic and it was about how Zoom was a videoconferencing app that was installing an http server (read, a security hole on the user’s computer) which remained even if you deleted the app. This was to “improve the user experience” so that it could quickly reinstall itself if you clicked one of their web widgets to start a call.
It’s worth checking in on exactly what software is doing in the background, auditing its activity and coming to a more precise understanding of 1. The reputation of the company behind it and 2. How they came to have this reputation and whether it is still relevant.
This is from July 2019: https://daringfireball.net/linked/2019/07/10/zoom
After they were called out, supposedly they fixed it, but that tweet you just linked looks like more of the same nonsense which goes right back to my original point: reputation matters. If the first could be taken as honestly naïve, the second proves it was not. Zoom doesn’t go on anything I own or control.
I understand 'mistakes' happen (though this was way too elaborate to be a mistake). But to just shrug it off and refuse to do anything when it's discovered is just total ignorance of security.
I'm not sure how that can be justified. Besides privacy, the issue is that this prevents the sound card from going to sleep, which may be an issue on laptops. But I guess this is insignificant compared to the rest of Teams' power consumption.
This stuff can drive you crazy. Each month there is some new annoyance or broken part, that I discover.
[1] I really liked to walk'n'talk for a few recurring meetings. Unfortunately, Microsoft does not like people touching grass.
I can understand not detecting something, or badly, but if it works now, and then on the next conference it figures "nah, there's no mic", I just can't understand what it does.
I used to think that this was an issue with me running Linux, and an "unsupported" distro at that (Arch). But I'm always reassured (in a way) when I see people having the exact same issues I do on Windows, with basic, run-of-the-mill configs (I have multiple sound cards, some of which come and go).
mic = grab_access_to_mic()
while app_is_running:
if (is_muted):
pass
else:
send_that_audio(mic)
Also, seems like an honest mistake, but I think they got this backwards, right?
> They used runtime binary analysis tools to trace raw audio in popular videoconferencing applications as the audio traveled from the app to the computer audio driver and then to the network while the app was muted.
Wouldn't it be driver -> app -> cloud? I think I'm splitting hairs at this point though.
Lastly, it would be nice if this article at least listed the apps that were investigated.
Yes, I work on an app that keeps the mic running all the time because of the above, and because ASIO doesn't allow disabling input at all.
Even if you get an external microphone which can be muted, if you're on a laptop you'll still have an internal microphone which can't be muted except through software.
What we really need are laptops sold without microphones and cameras. Then you can just use external ones only, and be sure that no one's listening/looking when you unplug them.
Much like the old physical typewriters had spyware hidden on them without the users knowledge. I would be willing to bet that every keyboard has something similar today.
It's also striking to see how many people believe that they can trust the 'little indicator lights' on their microphone and cameras to actually indicate that their physically cut off from power. Very, very few devices are made in a way that this is true, and I would be hard pressed to believe it even if it was started in a manual unless I physically checked the equipment.
Ultimately, just assume that the entire world can scrutinize everything that happens on or around any electronic device... so everything, all the time.
Another solution is to mute at the microphone, if your hardware has a button for that. This way the application can do whatever it wants, it will still get nothing. Using the hardware button is often less effort, than switching windows, finding that unmute button visually and moving the mouse to that button to click it. Or one could use push to talk. Since there are ways to mute yourself without having to do it in the app, it would be acceptable, if unmuting took a part of a second to be effective, indicating that by some "unmuting ..." label somewhere.
But still consider using both software and hardware mutes. I was on a sales call years ago and activated the hardware mute. While one of our salespeople was talking I groaned out loud, and the call suddenly went silent. Somehow the hardware mute had failed, despite the light being lit.
If it takes any more than 0 time, it's caused by badly written software.
I wouldn't blame the HW for the privacy issues, here.
They don't have to actually send the data though in this case they should just send 0 padding. It's all encrypted presumably, so the only externally observable factor is the packet size.
If you actually value privacy as a company though, these are all very solvable problems.
If you think they're doing something else, then don't use it. If you think you don't have a choice because your employer requires you to use it, your choice is not in whether or not to use the software. If it's something you care about, there's always a choice.
At a hardware level, grabbing the microphone can take time. Even worse that timing is inconsistent across devices, workloads, etc. That leads to a bad experience when unmuting and needing to delay your commentary. The solution to this is to keep the microphone on, but mute at a software level. This way the mic is always hot and ready to relay audio as fast as the software can switch.
I'd be somewhat willing to bet continue to stream audio is also a quality assurance mechanism. Some networks will shape traffic according to load. A quick jump in bandwidth can introduce unexpected jitter and latency. By continuing to stream audio (but not necessarily process or re-transmit), video conferencing can better ensure an un-interupted experience.
----
With that being said, if you really care about privacy, consider getting a hardware mute microphone.