Exactly. The ergonomics of much software is pretty damn terrible with users struggling to get done what they want to do. Thus, developers who rely on telemetry only cement-in bad problems.
It's for this and privacy reasons I always turn telemetry off.
It's very valuable feedback to hear, "wait, it does that?" Yeah, we spend a quarter million dollars on a feature that our users don't even know exists.
Sure it is. It's not a perfect indicator all the time, but can definitely pinpoint a common workaround users perform due to a lacking or unfriendly interface. It's just a helpful data point though, not a silver bullet.
I'm with you on the trust issues though when 90% of software companies abuse our data. Just another reason why we can't have nice things.
> In practice, I’m just doing workarounds to accomplish things the telemetry won’t explain on its own
This is one of the most obvious things people are looking for in analytics. People will always use your product in unpredictable ways to get the whatever functionality it is that they really wanted. This is one of the things product managers are most interested in knowing about because they either want to properly implement that functionality, or they already have it and they want to know if they’re doing a bad job of getting their customers to adopt it.
It’s incredibly unlikely that you’re such an ultimate power user that what you want out of the product is so unique that you’re inventing your own usage patterns that others aren’t also following.
I completely agree that you should have the right to not share your information with anybody you choose. But if you choose not to share any information about your use case with the company that makes the product, then you don’t have much right to complain about it not doing what you want it to.
Please don't cross into personal attack. You don't need to, it poisons discussion, and it evokes worse from others. Your post would be fine without that bit.
Watching individual user sessions from session recordings provides more depth, but you are still missing out on great swathes of information.
Analytics and session recordings don't give anything near as much concrete data as taking the time to engage with your users. You then use analytics to confirm behaviour.
Hum... On real life you would quickly discover how vulnerable all of your instruments are sensitive to both random and bad-intended interference.
You can apply the same phrase as a metaphor for software.
Well no, commercial pilots have to be rated IFR. Flying via instruments only is completely normal and part of the training. Visual is perhaps easier for many people, but you lose the use of visual navigation in all kinds of circumstances, like at night, over the ocean, or in a storm.
I generally disable the analytics in software I buy, but I do participate in surveys they send my way. Would that count for you?
Forums can also be a rich source of use case info for software developers who really care.
What you wrote here sounds very naive to me.
I might be completely wrong, but i assume if you have a large enough user base you would use telemetry only in an aggregated way, and then prioritize the majority of the user base. So I might totally be underrepresented but still have the same loss of privacy as everyone else.
I also feel i always have the right to complain about a product, publicly and vocally (it's just a different channel for analytics/telemetry, imo). Much like product owners have the right to just change it remove features that break my workflow.
> I might be completely wrong, but i assume if you have a large enough user base you would use telemetry only in an aggregated way, and then prioritize the majority of the user base.
In some cases yes, but you need to get that aggregate data from somewhere. It’s also naive (and probably a bit arrogant) to assume that your requirements aren’t in fact well aligned with the majority of the user base.
This is however a somewhat immature way of using analytics data. There’s plenty of incredibly valuable data you can get from peculiar and unpredicted usage patterns. Any product manager will tell you that most customers are quite bad at expressing what they actually want. A good product manager can gather a fair amount of insight from what they choose to complain about, but analysing the most immediately obvious trends really only gets you so far. You’ll often get the most insight from the rare customers who are able to concisely articulate their feedback, and those that are frustrated and ingenuitive enough to subvert your application to their will.
For any motivated organisation, there’s countless ways to get insight out of their analytics. You’ve just described the lowest hanging fruit.
> I also feel i always have the right to complain about a product
Depending on where you live, you likely have the right to complain about anything you like. But if you want to prevent an organisation from analysing your usage of their service, then complaining about them not meeting the needs of your use case is hypocritical.
No. In any high-dimensional space (in this case the dimensions are features used by a given user), the vast majority of the elements will be outliers in at least one dimension. Everyone has idiosyncrasies. It is arrogant to assume that everyone is the same as oneself.
The problem is those companies view it as their data for which you have no right to modify it. Let alone chose to transmit it at all.
Before anyone replies this would invalidate telemetry, ask yourself this question: "If you see a spike in data that makes no sense, isn't that an indication that somehow your data collection policies are pissing off people?"
I really don't have the time or malice to do this. I can and do move on to something else.
Giving a business your data is an act of charity. It should never be an expectation.
Yes, it costs more to collect the data manually. It usually costs more to behave ethically when you aren't legally required to.
> Yes, it costs more to collect the data manually.
Not just that, but you also get data biased to people willing to spend two hours answering questions for a $25 Amazon gift card.
Sure, but the whole point of this article was that the telemetry data is also biased.
There is no magic bullet. Your options are to get biased data the cheap way or the ethical way. Either way, the data is biased and you have to consider that when making decisions.
It doesn't just ethics, and I think it should be "first, do no harm", as in, first, be ethical.
Btw, opt-in metrics are ethical and cheap afaik. As for biased? all metrics are biased...
A business is not obligated to improve a service just because you donated your telemetry data.
If they stop improving, I'll stop paying them as well as stop giving them my data.
Alternatively the business could pay us to do that but obviously they resist the idea.
Not all software telemetry is that creepy, but things like google analytics definitely are.
Much clearer.
It’s not as big of a deal now that most users upload everything anyway.
Are you suggesting companies spend 10x on manual user research? Are you willing to pay more for a product because of increased costs to understand what users actually want?
Besides, When we bought magazines the editors didn’t know what stories people were reading or what words you were spending more time reading. When you buy a tool the manufacturer doesn’t know how you use it. Why software gets a free pass at getting all that data for free without asking is beyond me. What I expect are for regulations to eventually hit this industry.
this is the reality of how businesses operate.
> product pricing and labor aren’t that tightly correlated.
If you're the staff accountant, sure. You're technically correct.
If you're the CFO and I'm trying to convince you we need to spend $1M on user research instead of $100K, you can be sure this is taken into account when modeling monetization strategies to recoup R&D costs.
It's the reality of how a subset of industry operates, which doesnt make it less wrong.
Move your analytics to the server and stop tracking my every mouse move and page scroll. Needing to know that stuff suggests you're either creepy or that your organisation is saturated with marketing and SEO wonks.
User research and analytics are two entirely different things. Your product will suffer if you neglect the first.
if they want to make their product better, that's what they should do, instead of drawing wrong conclusions from the limited data that telemetry gives you.
> Are you willing to pay more for a product because of increased costs to understand what users actually want?
It's your business. Add telemetry, have it turned off by a significant amount of your users which might not be an average representation of your user base.
So your go ahead and optimize for a biased subset, maybe even interpret some of that data wrong, and before you know it, for iterations later some fancy startup is stealing the show, because they simply read all the detailed complaints about your software on reddit and HN instead of jacking off to analytics data.
Just curious, what's so inexcusable about that?