>I'll play the devil's advocate.

Your point is basically "surveillance data is useful". And, well, yes. There would be zero debate over surveillance if there were literally no desirable reasons to have it.

Calling it surveillance is misleading in most cases.

The most recent crash reporting system I worked with was little more than a stack trace without any user data recorded at all. Not even IP addresses of the report.

We didn’t care who was crashing and we didn’t collect any PII at all. It was a simple report about where in our code the crash occurred.

It was very useful for fixing bugs. No surveillance or PII involved.

If that was the norm then people would opt in. But the trust is gone now, bad actors have ruined it for everyone. And the solution to that is not to enable it by default.

I think this might be the winning argument. There may not be any meaningful reason for telemetry to outweigh the bad actors and damage they've done.

For me, i'd love to enable telemetry for some of my more liked, FOSS apps - but even with those my question immediately arises "What are you sending?".

Without someway to monitor, are they sending filenames? Are they sending file contents? How much is it? etcetc

To satisfy my questions i need some sort of router-enabled monitoring of all telemetry specific traffic. So i can individually approve types of info... and that seems difficult. But the days of blanket allowances from me are long gone due to the bad actors.

Excellent point. As a user, here are my requirements if you want me to opt into your data collection scheme:

1. All exfiltration of data must be under my direct control, each time it happens. You can collect all the data you want in the background, but any time it is transmitted to the company, I must give consent and issue a command to do it (or click a button).

2. All data that is exfiltrated must be described in detail before it is exfiltrated. "Diagnostic data" isn't good enough. List everything. Stack trace? Crash report? Memory dump? Personal info (list them all out)? Location information? Images (what are they, screenshots? from my camera?) Time stamps from each collection. If it's nebulous "feature usage data" then list each activity that is being logged (localized in my language). Lay them all out for me or I'm not going to press that Submit button.

3. I need to be able to verify that #2 is accurate, so save that dump to disk somewhere I can analyze later.

4. The identifiers used to submit this data should be disclosed. Is a unique user id required to upload? Do you link subsequent uploads to the same unique id? Is that id in any way associated with me as an account or as a person in your backend? I want you to disclose all of this.

5. For how long do you retain the data I sent? How is this enforced? Is there a way for me to delete the data? How can I ensure that the data gets deleted when I request it to be deleted?

6. Do you monetize the data in any way, and if so, am I entitled to compensation for it?

I don't know of many (if any) data collection schemes that meet this bar, yet.

I'm fairly sure Android crash reporting is all manually done, and even zips it up for you to inspect.

But some of these are impossible, e.g. how do you localize a stack trace?

If you install an app via Google Play, crash reporting & some telemetry are silent and enabled by default. This is in addition to any crash reporting built into the app.

https://support.google.com/accounts/answer/6078260

Analogy time: in the early days of the Internet, any IP address could send an e-mail by directly connecting to the mail change host listed for the domain of the To: address via SMTP. This was a good, nice thing.

Bad agents, spammers, ruined that. The trust is gone, and wow it's a terrible idea to accept SMTP connections from any random IP addresses.

Legitimate senders have to "opt in" to the SMTP sending system by getting a static IP of good repute, or else using a forwarding host which has those attributes.

For ad industry, trust is gone now. But for telemetry? I think most people don't much care.

Until you realize that most telemetry infrastructure is provided by ad companies (Google Analytics for example).

That, and additionally, if there's an opportunity to monetize data from (or derived from) telemetry further, companies won't hesitate selling it (or access to it) to third parties.

That, and additionally, if the data is stored, it's a question of when, not if, it'll become a part of some data breach.

To developers and entrepreneurs feeling surprised by the pushback here, please take a honest look at the state of the software industry. Any trust software companies enjoyed by default is gone now.

They don’t care because they don’t know.

Most people have no idea what "the norm" or "telemetry" even is so I highly doubt this would affect opt-in rates at all.

I think the old crash report screen will be answer to the paranoid crowds now. App crash the stack trace windows appears with the data and a button to click to email the stack trace to the developers. That used to be the norm before privacy data became lucrative business.

It sounds like there must be a standard of privacy for certain apps.

I work with UL a lot and they have lists of standards and specifications that help us meet the safety requirements of electronic devices. These standards are then used to meet the customers demand for a high level of safety. Customers in my field do not even consider products that don’t have UL. This strategy is good to better inform the consumer while standards are kept by an independent firm whose incentives are aligned to maintain their credibility.

I am not deep in the software field but I can imagine that groups like the EFF or similar orgs have a standard. The issue is that the consumers of these products don’t seem to care about this outside of the privacy advocate world.

In Linux, we use `abrt` for that, but user need to review and submit bug report with stack trace manually, so nobody complains about that. Yep, it's very useful.

I think that is red hat specific. I have seen it on fedora but not arch or gentoo.

How do users know if that's all the data that's submitted? Auditing every program to see exactly what gets sent (and repeating the process for every update) is way too much work; it's safer to just opt-out by default.

Now that I think about it, it's safer to simply not use software that opts users into studies (like telemetry analysis) without informed consent.

> How do users know if that's all the data that's submitted?

That's the thing, isn't it? They'll never know. They can't; it takes deep technical knowledge to even be able of conceptualize what data could be sent, and how it could be potentially misused.

Which is to say, it's all a matter of trust. Shipping software with opt-out telemetry (or one you can't disable) isn't a good way to earn that trust.

Even with deep technical knowledge and severely pared down telemetry, PII embedded in side-channel like outlets could be missed. Think a simple stack trace with no data is PII-free? Probably. But are you sure that the stack of functions called doesn't depend on the bytes in your name?

Stack traces can include private user data too, I always review all of the data in a stack trace before filing bug reports.

Does removing PII preclude data from being surveillance?

The anti-tracking point in the era of Do-Not-Track was “building advertising profiles that follow people around the web is evil.” Software telemetry is neither a personal profile, nor for advertising, nor correlated across more than one surface. As such I think it’s a little bit of a stretch to put this in the same bucket as 3rd party JS adtech crap.

> Your point is basically "surveillance data is useful". And, well, yes. There would be zero debate over surveillance if there were literally no desirable reasons to have it.

Well, not exactly. The original comment was that "surveillance data" is useful to the user and their installation. For instance in getting better response to your error report. Or, I'd say, in keeping your software patched and up to date (check for updates is included in the OP as suppressed by 'do not track', since it necessarily reveals IP address).

Even if none of these things that make it useful to the actual users were in effect... surveillance would still be happening because it lets the vendor monetize the users data.

Useful to whom and for what seems relevant, instead of just aggregating it all as "sure, it's useful in some way to someone or it wouldn't happen."

> For usage data, it allows developers to focus on features that matter and know which ones you can remove.

I find tools that don't do this are generally more powerful because they allow for deep expertise and provide a ton of payoff if you put in the effort.

E.g: Vim. 80%+ of users probably don't use macros. Hell, I use them <1% of the time. But I'm sure glad they're there when I need them.

20% of users is quite a lot. In most mature applications you can find a lot of painful code twisted around requirements for 0.1% of the users or less.

> and know which ones you can remove.

No, you can't remove it. Even though I'm using it rarely it's existence might be the reason for me to use the tool at all, so that when I need it the feature is available.

This came about with audacity. There I have my set of standard filters I run all the time, even though they don't bring much benefit, they are there and nice. They will be on top of a usage statistics. Then there are filters I need for a special effect or to repair something really broken. Those I use ahrdly, but when, they make the difference.

Or when talking command line: `ls` without options I use a lot (Well actually a lie, i have some alias in my shell rc), sometimes I use `-a` or `-l`. This doesn't mean that maintainers should remove `-i` since once per year or so I need it to compare inodes with log entries or something and then it's important that flag exists.

You need qualified information about what features are important. Not unqualified statistics.

Ah, so the question of whether to remove feature A or feature B is solved by simply not removing either. That's brilliant!

I get that this might not be a popular sentiment, but resources are finite. If we have a situation where we can't maintain both features, which one do we focus on? Usage metrics can absolutely be beneficial there.

What you need is qualified metrics. Like asking your users what is important.

Also relevant is: Is there a maintenance cost or is that code that rarely needs attention and doesn't compete with important new work.

I don't think it's usually a problem of "remove either A or B", it's usually "should we remove A or invest work in making it compatible with changes ahead?".

I see how usage telemetry can be useful in deciding whether or not it's worth it to keep supporting a feature, but I offer two counterbalancing points:

1. What people may be worried about - what I myself am worried about - is the methodology creep; it's too easy to end up having telemetry drive feature removal decisions, as in, "monthly report says feature X is used by less than 1% of users, therefore let's schedule its removal for the next sprint". The problem here is, telemetry alone will likely lead you astray. It's useful as a data source, not as the optimization function of product development.

2. If a feature you're worrying about has significant use, you most likely already know it without telemetry - all it takes is following on-line discussions mentioning your product (yes, someone might need to do it full-time). If removing the feature will have major impact on your maintenance budget, and non-telemetry sources don't flag this feature as being actively used, you can just axe it - revenue hit from lost userbase you've missed is unlikely to be big.

From this follows that the telemetry is most useful for deciding the fate of features that aren't used much, and don't cost much to maintain. At which point, I wonder, do you really have such low margins that you can't afford to carry the feature a while longer? I'm strongly biased here, because I'm going only by my personal experience - but I'm yet to see a software company[0] that doesn't have ridiculous amounts of slack. Between the complexity, management mess-ups, piles of technical debt and the nature of knowledge work being high-variance, having a feature slow your current development down by half[1] won't have much long-term impact.

--

The question thus is, are the gains from usage telemetry really worth the risk and potential ethical compromise? Would those gains be significantly lessened, if the telemetry was opt-in, and the company put more work into getting to know the users better? I suspect the answer is, respecting your users this way won't hurt you much, and may even benefit you in the long run.

--

[0] - Other than one outsourcing code farm I briefly worked in (my boss loaned me for a couple weeks to a friend, to help him meet a tight deadline), but these kind of companies don't make product decisions, they just close tickets as fast as possible.

[1] - And hopefully leading some devs notice the need for a refactoring, in order for that feature to not be a prolonged maintenance burden.

I'm a hundred percent certain that 1 is what Spotify does. It seems every release they remove or hide features that I use occasionally (e.g. clicking on the currently playing album picture to reveal the active playlist). It's extremely frustrating that since power users now are in a minority they are completely ignored. At the first release like this they basically said "This new version has a lot less features, but you can vote on which ones we will add back!". They added none of the ones that got votes, and eventually removed the feature voting system altogether.

> Most people will shoot support e-mails at you which are more or less "app crashes". (...) If you collect crash reports, you have probably already fixed the problem.

Fair enough. Still, there are two separate steps here: collecting crash reports and sending them. What if the app asked if it can send the report, letting you optionally review it? Many programs today do that, I think it's an effective compromise. Additionally, the app could store some amount of past crash reports, and the places for the users to get the support e-mail (a form, a button, an address in a help file...) could request you to check for, or automatically call up, those past crash reports, and give the user choice to include them. The way I see it, the app should give users near-zero friction to opt-in, but still have them opt-in.

It won't solve the problem of bad support requests completely, but nothing ever does - random people will still write you with problems for which you have no data (e.g. network was down when crash occurred), or for which no data exists (because requester is a troll).

> For usage data, it allows developers to focus on features that matter and know which ones you can remove.

I accept this as an argument in favor, though personally, I don't consider it a strong one. I feel that "data-driven development" tends to create worse software, as companies end up optimizing for metrics they can measure, in lieu of actually checking things with real users, and thus tend to miss the forest for the trees.

Picking good metrics is hard, especially in terms of usage. The most powerful and useful features are often not the ones frequently used. Like, I may not use batch processing functionality very often, but when I do, it's critical, because it lets me do a couple day's worth of work in a couple minutes.

So, for me, can usage telemetry improve software? Shmaybe. Is it the only way? No, there are other effective - if less convenient - methods. Is the potential improvement worth sacrificing users' privacy? No.

> on iOS crash reports are opt-in, and only about 20-30% of users have them enabled. That is fine for huge programs or ones with little surface.

I feel the main reason this is a problem is because of the perverse incentives of app stores, where what you're really worried about is not crashes, but people giving you bad reviews because of them. Mobile space is tricky. But then, forcing everyone into opt-in telemetry doesn't alter the playing field in any way.

Or how about compensating the user when their software crashes?

It's not such a strange thing given that they paid for the software.

And if sending a crash report means receiving money, then more users will send them.

This would be tricky - the nature of software means some people would try to keep crashing their programs on purpose, sending repetitive crash reports, in order to make money. Developers would now have to deal with a flood of spam in their crash reports.

I think the only way crash reporting can work, outside of support contracts, is as a favor by the user to the vendor. But, to maximize the amount of such favors, the vendor would have to treat users with respect - which is pretty much anathema to the industry these days.

Valid concern, but bug bounties are a thing; It’s up to the developer to decide if the bug is worthy of a payout. Maybe make it so that if it crashes, and they provide a useful log (or steps), then you pay out.

From what I know, bug bounties already have a spam problem. I definitely saw some devs in my circles complaining about people repeatedly sending garbage submissions in hopes of getting a payout.

What bug bounties also have is a big barrier to entry. You generally need to be at least marginally competent in software development, and do plenty of leg work, to make money with them. Turning regular crash reports into bug bounties removes that barrier, amplifying the spam problem.

One point here - support emails do not help you identify problems that less invested users may be having with your product.

For example, I used to work developer relations on TensorFlow. We wanted to make the framework accessible to enterprise data scientists. The problem was that these users were not familiar with the tools that we commonly used to get feedback - GitHub issues, the mailing list, etc.

Most of them were using TensorFlow on Windows via Jupyter, which wasn't well-represented among the users that we had frequent contact with.

It was really hard to understand the universe of issues that prevented most of these users from getting beyond the "Getting Started" experiences. Ultimately, these users are better served by easier to use frameworks like PyTorch, but I think a big reason that TensorFlow couldn't adapt to their needs is that we didn't understand what their needs were.

Another big problem is that it takes a certain level of technical sophistication to know how to send maintainers a useful crash report. If you rely on this mechanism, you will have a very biased view of your potential user base.

Sure, and I can spy on girls I like just so I can learn how to offer them a better experience when they meet me.

Having good intents does not justify skipping consent. The “opt-out” mentality is a very slippery slope, since you’re already stating that consent does not need to be explicit (hint: it’s not consent if not given explicitly AND freely).

The similie fails since software tracking in programs you run are agreed to by you when you accepted the license agreement.

Yet another reason to prefer free as in freedom software, which can be forked.

I don’t think the simile fails for that reason. “Informed consent” is generally accepted as the standard, and it’s arguable that click-through EULAs are not in any way “informed consent” for the average person.

To bring the original simile full circle, let’s say a person signs a contract with another person, and then that person forces themselves on them. I doubt the presence of a clause in the contract saying “I agree to have sex with X” would absolve them of guilt.

There certainly is a difference in harm between forcing sex and sending unique user IDs, and it’s not unreasonable to make a contract about the latter if one wishes, is it?

This is rather a reason to prefer free software licenses. Culture has yet to catch up to this, but in the long run I hope the collective consciousness learns to distrust and avoid complicated proprietary software licences.

Why not collect the data locally and only share it after a crash when the user agreed?

Why not add a splash screen on the program start up that informs your user of upcoming plans so they can intervene? Like "Hey we are planning to remove feature X to speed up the program, do you agree?"

And this is because actual usage metrics don't really translate to opinions. I have features in programs that I use maybe once every two years, but then I really need them. Then there are other features I use daily and I really hate them with a passion.

That certainly is the devil’s side. Unfortunately too many firms have affixed phony halos and then exfiltrated the People‘a personal data. Opt-in is the only way the People will be able to choose whom they trust.

Why not do local logging by default (opt-out), and ask users to upload their logs if and when they want you to diagnose some problem?

Telemetry alone doesn't tell you how valuable a given functionality is. A critical problem, one imposed through the tyranny of the minimum viable user,[1] is that the high value creator community of any tool or service is going to be small. Quite often 1%, if a very small fraction of that.

Your telemetry data will show you only that such features see little use. They won't tell you how much value is derived from that use, or what the effects of removing such functionality will be on the suitability and viability of your application.

Default telemetry is demonstrable harm for negative gains.

________________________________

Notes:

1. See: https://old.reddit.com/r/dredmorbius/comments/69wk8y/the_tyr...

(disclaimer: IANAL)

You have basically just given a justification that crash reporting can be conducted based on legitimate interest instead of consent, and as such does not require opt-in.

Many people mistakenly believe consent is the only possible justification for data processing under GDPR, whereas there are actually 6 possibilities, and you can ask a lawyer which one can apply for a given data processing flow.

Note that whereas I do believe that crash reporting can indeed be considered legitimate interest, I wouldn't consider plain telemetry ("phone home without a technical good reason") to fall under that umbrella...

Not everyone is using a flatrate and thus both crash reporting and telemetry might cost the sender some money (or calculate against their hugh speed quota). Some people seem to expect that data transfers are always free ...

That's one reason (besides privacy) why I have Netguard running as a firewall on our Android phones and set to block traffic by default for each app, unless the app's creator convincingly explains why their app should be allowed to access the net.

One should not simplify the work of a group people at the price of the entire community.

Actually, this is why I want telemetry to be opt-in. I have a consistent policy of providing telemetry and I want the software to be biased to my needs. I want them to conclude that some feature used only by some privacy-conscious user is unused and should be axed because I want the software to be hyper-tailored to me.

I want the software to be streamlined, have no features except what I'll use, and for the community to be specifically people like me. I want other people to not use the software and use up dev bandwidth.

And I love it when telemetry biases the stats towards me. That way all devs will eventually be making software for people just like me.

I know you're sarcastic, but it wouldn't be a bad outcome. Sure, the vendor would have to be particularly dumb in their usage of telemetry[0], but the result would be... software that is useful to you. All the professional features you need would be in there, with none of the glitter.

Of course, I would opt-in too, with the same mindset but different use cases, and the software would provide equally for both of us. Add in a few more people like us, and we'd end up with a quality tool, offering powerful and streamlined workflows. Those who don't like it would start using a competing product, and tailor it towards their needs. Everyone wins.

Reality of course is not that pretty, but at face value, it still beats software optimized to lowest common denominator, serving everyone a little bit, but sucking out the oxygen from the market, preventing powerful functionality from being available anywhere.

--

[0] - It's a mistake that's much easier to make when you're flooded with data from everyone, rather than having a small trickle of data from people who bothered to opt in.

I am actually not being sarcastic. It is a life goal for me to have most policy organized to serve me or people like me (along whatever genetic/social/cultural/economic grouping is most likely to benefit me). i.e. I encourage communities not like me to refuse participation in medical research, forcing participants to be in my ethnic group; I encourage stringent data sharing norms and a culture of fear around what is done with health data in socio-economic and ethnic groups that are not mine; I encourage organizations to have strict opt-in requirements, in general, which I have no problem meeting, so that tools are built to be best used by me and adequate for others.

My dream is that everything is above the adequacy threshold for everyone else so that they don't build their own equivalent tool but that everything is also past my pleasantness threshold. I think the most effective means of doing this is to focus existing products into being past my pleasantness threshold while ignoring others since high switching costs keep most people on the same path they were before, and because things like medical research they don't really get to re-optimize.

I understand that this sounds sarcastic, but it is not.

Well... then I apologize for assuming. I'm not sure how such philosophy sits with me, I need to think about it more. One thing for sure, what you say makes you the model of an ideal free market participant :).

And I'll play the devil's devil's advocate:

For Support: Follow these steps:

Step 1 - opt-in to telemetry ( diagnostic ) data reporting

Let's not beat around the bush, data costs money. Personal data costs money.

How much are you willing to pay me for my app usage data? Oh, nothing? Well then, buzz off.

Literally all the software in question is free. Yeah, you can both tell each other to buzz off and no one will be upset.

http://i.giphy.com/Qp45SP1P3PbbO.gif

If that were the case, why do Microsoft apps still crash so frequently

My experience is third party products/add-ins (specifically McAfee Enterprise Suite) cause most of the crashes I have with Office products. Most OS blue scrrens are non-microsoft hardware drivers.

I was mainly referring to their apps on Mac and Android.. They crash a LOT. Especially Teams for Mac and Outlook on Android. And both use lots and lots of telemetry. Lots and lots of traffic to hockeyapp.com (which is a MS telemetry platform) and various telemetry URLs at microsoft.com.

> know which ones you can remove

I know the answer to that question. None. Never remove a feature.

Yeah, I understand this becomes necessary to minimize maintenance effort on rarely used features,but it feels like even more often it's used to turn useful applications into barren wastelands. Firefox, perhaps the best example, is a fraction of what it used to be.

This may be a noble goal but then it may also lead to the program never being updated. If you need to do a major update (underlying system library no longer works, dependency is deprecated and has a security hole, a new direction requires overhaul of backend) you may need to prioritise what to keep and what to let go.

A problem I encountered was also localisation. Once you localise your program, adding any string is exponential work. In this case removing features can give you a lot of slack.

> For usage data, it allows developers to focus on features that matter and know which ones you can remove.

Sorry, but if devs are requiring THAT tight connection with end users to MAINTAIN software, they are probably should stop and leave. Its impossible to figure out a new feature from the such reactive approach, and they would have to resort to more traditional way to interact with end users. Thus... making coverage analysis a totally redundant thing.

Tighter user connection is suitable for enterprise software, not for general deployment.

And why so much worries about removing working(!) features?