Then again most browsers have a print button.
Every time a new rule was added, you had to wait up to 24 hours for the css servers to propagate.
You have to delete something every once in a while to make room for something new.
If you want to read the data behind many of the decisions, read these papers [0].
Just curious, what's so inexcusable about that?
Exactly. The ergonomics of much software is pretty damn terrible with users struggling to get done what they want to do. Thus, developers who rely on telemetry only cement-in bad problems.
It's for this and privacy reasons I always turn telemetry off.
It's very valuable feedback to hear, "wait, it does that?" Yeah, we spend a quarter million dollars on a feature that our users don't even know exists.
Sure it is. It's not a perfect indicator all the time, but can definitely pinpoint a common workaround users perform due to a lacking or unfriendly interface. It's just a helpful data point though, not a silver bullet.
I'm with you on the trust issues though when 90% of software companies abuse our data. Just another reason why we can't have nice things.
> In practice, I’m just doing workarounds to accomplish things the telemetry won’t explain on its own
This is one of the most obvious things people are looking for in analytics. People will always use your product in unpredictable ways to get the whatever functionality it is that they really wanted. This is one of the things product managers are most interested in knowing about because they either want to properly implement that functionality, or they already have it and they want to know if they’re doing a bad job of getting their customers to adopt it.
It’s incredibly unlikely that you’re such an ultimate power user that what you want out of the product is so unique that you’re inventing your own usage patterns that others aren’t also following.
I completely agree that you should have the right to not share your information with anybody you choose. But if you choose not to share any information about your use case with the company that makes the product, then you don’t have much right to complain about it not doing what you want it to.
Please don't cross into personal attack. You don't need to, it poisons discussion, and it evokes worse from others. Your post would be fine without that bit.
Watching individual user sessions from session recordings provides more depth, but you are still missing out on great swathes of information.
Analytics and session recordings don't give anything near as much concrete data as taking the time to engage with your users. You then use analytics to confirm behaviour.
Hum... On real life you would quickly discover how vulnerable all of your instruments are sensitive to both random and bad-intended interference.
You can apply the same phrase as a metaphor for software.
Well no, commercial pilots have to be rated IFR. Flying via instruments only is completely normal and part of the training. Visual is perhaps easier for many people, but you lose the use of visual navigation in all kinds of circumstances, like at night, over the ocean, or in a storm.
I generally disable the analytics in software I buy, but I do participate in surveys they send my way. Would that count for you?
Forums can also be a rich source of use case info for software developers who really care.
What you wrote here sounds very naive to me.
I might be completely wrong, but i assume if you have a large enough user base you would use telemetry only in an aggregated way, and then prioritize the majority of the user base. So I might totally be underrepresented but still have the same loss of privacy as everyone else.
I also feel i always have the right to complain about a product, publicly and vocally (it's just a different channel for analytics/telemetry, imo). Much like product owners have the right to just change it remove features that break my workflow.
> I might be completely wrong, but i assume if you have a large enough user base you would use telemetry only in an aggregated way, and then prioritize the majority of the user base.
In some cases yes, but you need to get that aggregate data from somewhere. It’s also naive (and probably a bit arrogant) to assume that your requirements aren’t in fact well aligned with the majority of the user base.
This is however a somewhat immature way of using analytics data. There’s plenty of incredibly valuable data you can get from peculiar and unpredicted usage patterns. Any product manager will tell you that most customers are quite bad at expressing what they actually want. A good product manager can gather a fair amount of insight from what they choose to complain about, but analysing the most immediately obvious trends really only gets you so far. You’ll often get the most insight from the rare customers who are able to concisely articulate their feedback, and those that are frustrated and ingenuitive enough to subvert your application to their will.
For any motivated organisation, there’s countless ways to get insight out of their analytics. You’ve just described the lowest hanging fruit.
> I also feel i always have the right to complain about a product
Depending on where you live, you likely have the right to complain about anything you like. But if you want to prevent an organisation from analysing your usage of their service, then complaining about them not meeting the needs of your use case is hypocritical.
No. In any high-dimensional space (in this case the dimensions are features used by a given user), the vast majority of the elements will be outliers in at least one dimension. Everyone has idiosyncrasies. It is arrogant to assume that everyone is the same as oneself.
The problem is those companies view it as their data for which you have no right to modify it. Let alone chose to transmit it at all.
Before anyone replies this would invalidate telemetry, ask yourself this question: "If you see a spike in data that makes no sense, isn't that an indication that somehow your data collection policies are pissing off people?"
I really don't have the time or malice to do this. I can and do move on to something else.
Giving a business your data is an act of charity. It should never be an expectation.
Yes, it costs more to collect the data manually. It usually costs more to behave ethically when you aren't legally required to.
> Yes, it costs more to collect the data manually.
Not just that, but you also get data biased to people willing to spend two hours answering questions for a $25 Amazon gift card.
Sure, but the whole point of this article was that the telemetry data is also biased.
There is no magic bullet. Your options are to get biased data the cheap way or the ethical way. Either way, the data is biased and you have to consider that when making decisions.
It doesn't just ethics, and I think it should be "first, do no harm", as in, first, be ethical.
Btw, opt-in metrics are ethical and cheap afaik. As for biased? all metrics are biased...
A business is not obligated to improve a service just because you donated your telemetry data.
If they stop improving, I'll stop paying them as well as stop giving them my data.
Alternatively the business could pay us to do that but obviously they resist the idea.
Not all software telemetry is that creepy, but things like google analytics definitely are.
Much clearer.
It’s not as big of a deal now that most users upload everything anyway.
Are you suggesting companies spend 10x on manual user research? Are you willing to pay more for a product because of increased costs to understand what users actually want?
Besides, When we bought magazines the editors didn’t know what stories people were reading or what words you were spending more time reading. When you buy a tool the manufacturer doesn’t know how you use it. Why software gets a free pass at getting all that data for free without asking is beyond me. What I expect are for regulations to eventually hit this industry.
this is the reality of how businesses operate.
> product pricing and labor aren’t that tightly correlated.
If you're the staff accountant, sure. You're technically correct.
If you're the CFO and I'm trying to convince you we need to spend $1M on user research instead of $100K, you can be sure this is taken into account when modeling monetization strategies to recoup R&D costs.
It's the reality of how a subset of industry operates, which doesnt make it less wrong.
Move your analytics to the server and stop tracking my every mouse move and page scroll. Needing to know that stuff suggests you're either creepy or that your organisation is saturated with marketing and SEO wonks.
User research and analytics are two entirely different things. Your product will suffer if you neglect the first.
if they want to make their product better, that's what they should do, instead of drawing wrong conclusions from the limited data that telemetry gives you.
> Are you willing to pay more for a product because of increased costs to understand what users actually want?
It's your business. Add telemetry, have it turned off by a significant amount of your users which might not be an average representation of your user base.
So your go ahead and optimize for a biased subset, maybe even interpret some of that data wrong, and before you know it, for iterations later some fancy startup is stealing the show, because they simply read all the detailed complaints about your software on reddit and HN instead of jacking off to analytics data.
It tracked this, of course, by assuming that the mail would be opened in a web browser that would make requests for images.
He cancelled that decision:
"Logged opens for each newsletter are between 53% and 60% -- but an experiment a while back revealed that hundreds of you aren't logged, for various reasons around email security and that one guy who reads everything on a 1980's greenscreen monitor. Clicking of links depends massively on what's in the newsletter, plus the note in the previous sentence, but has gone as high as 41% of readers on one week this year."
Authentic question: how would a business know this otherwise in an actionable and effective way?
Maybe two cohorts. The people we're really sure about, and the ones we're pretty sure about. If they diverge, ask why.
Note that reCAPTCHA thinks you’re a bot if you block analytics…
The Textarea Cache Firefox extension https://addons.mozilla.org/en-US/firefox/addon/textarea-cach... is incredibly useful, but I keep forgetting to install it when I get to a new machine. I forget only once.
If you're running anything with an API, then unless somethings horribly wrong it's even easier: look at the number of requests being made to an API endpoint and spot check a few of the user identifiers (tokens, keys, whatever you're using) to see the variety of users.
All of this is assuming you're trying to merely investigate the volume of use of a feature, not trying to diagnose demographics. If you're trying to extract more fine-grained detail, I don't have as many answers; I hope others will chime in with constructive ways to get things like geographic demographics via server logs.
I don't use Google analytics but I have seen time and again vocal users who seem ignorant of their own usage of an application...and very much not representative of the majority.
Plus, people often don't give valuable feedback when asked questions about features they want and use... people are poor judges of what they actually want, and will list things they think they care about and then end up never using or which don't affect their choices.
We do. We have real customers do real surveys of our real web sites. In person. We even do shadowing to see how real people use our sites in their daily work.
It's uncommon in SV due to laziness and an unwillingness to talk to actual human beings. Which is dumb because there are companies that will handle this for you.
But SV is stuck in this mindset that everything can be solved by an algorithm. It can't. The tech echo chamber really needs to get over itself.
Our customers certainly do. We get excellent results from asking a few simple questions now and then, providing both a good source actionable feedback on feature requests and any current problems, and often some encouraging comments that reassure us we are basically doing things that our customers like.
It doesn't just have to be surveys with lots of participants, though. For example, we've known for decades that a simple observational study with just a handful of people is often enough to identify most of the serious usability problems with an interface.
The idea that everything important must be reduced to automated analytics and number-crunching is a very strange disease. Even if the numbers don't lie -- and as we see here, that is far from guaranteed -- you still need to be asking the right questions and comparing useful alternatives for the results to be valuable.
Sorry, your comment just reminded me of that. Are surveys perfect? No, but they have their uses, and plenty of companies find real value by making use of them.
Some folks seem to think analytics is the only way to get feedback from users.
Just ask me. Survey, email me directly, anything but siphoning all my data off to Google in the process.
Just searched "nginx 1st party analytics"
You can't just follow analytics. You have to understand your users.
It's true though. If you want to be recognized, you can't be incognito. That's like refusing to vote and then complaining about politics.
Chrome's analytics ought to say nobody uses Incognito Mode, but they'd be dumb to remove it.
Inform your users honestly and do not hide things. Make promises about tracking data usage and stick to them. Tell them the truth and give them a choice. Stop acting like you got something to hide and do your tracking yourself.
Oh, and by the way, responding to these kinds of issues with “if you wanted your voice to be heard, you should have turned on analytics” is inexcusable.