News for hackers. Stuff that matters.
Re:Telemetry in the Go Toolchain by marginalia_nu on Wednesday February 08, 2023 @03:09PM (#34714166)
> The idea of telemetry is being able to prioritize the work that will be most widely useful.

It does sort of hinge on the highly suspect assumption that usefulness is correlated with use. An obvious counter-example to this is something like a fire-extinguisher, which will in the ideal case just sit on a wall until it's use-by date passes and then it's discarded having never been used; or on the flip side, an incredibly byzantine workflow that could be reduced to something much simpler will appear important and useful.

Even without these edge cases, interpreting statistics is really hard. Like people with PhDs who have studied these things for years still get them wrong all the time.

What ends up happening more often than not is it's used as a tool to quiet the critics when pushing through unpopular changes.

Re:Telemetry in the Go Toolchain by cube2222 on Wednesday February 08, 2023 @03:22PM (#34714391)
All this boils down to "an unskilled engineer will misinterpret data even if they have it". I'll assume the Go team knows what they're doing, based on their track record so far.

There's a lot of very simple questions you can answer very reliably, too, like "what proportion of the users are still using a certain compatibility flag".

Re:Telemetry in the Go Toolchain by marginalia_nu on Wednesday February 08, 2023 @03:27PM (#34714480)
My point is that scientists whose job is to interpret data and construct experiments gets this wrong on a regular basis, despite years of training in constructing experiments and interpreting data, despite peer review, despite staking their career and reputation on not making these kinds of mistakes. They still happen! A lot!

Interpreting data is very hard.

Re:Telemetry in the Go Toolchain by masklinn on Wednesday February 08, 2023 @04:44PM (#34715715)
> I'll assume the Go team knows what they're doing, based on their track record so far.

Funny, I’d assume the exact opposite. After all much of the understanding of privacy and statistics at scale was developed after 1980.

Re:Telemetry in the Go Toolchain by philosopher1234 on Wednesday February 08, 2023 @08:14PM (#34718360)
Normally Hacker News is ruthless with stupid comments like this, but some sort of unexamined feelings of inadequacy make Go devs a fair target. I hate this attitude.
Re:Telemetry in the Go Toolchain by remram on Wednesday February 08, 2023 @08:27PM (#34718475)
Go is actually well known for dubious choices they reverted later, like not using libc, their weird major version scheme, or the absence of generics. There's a good record of charging ahead against the popular wisdom and being proven wrong.
Re:Telemetry in the Go Toolchain by nine_k on Wednesday February 08, 2023 @03:30PM (#34714516)
Most software features are not like fire extinguishers.

More than that, the interesting stats may be not even around user-visible features, but around internal mechanisms, like some cache hit rate, or how often is some branch in the compiler invoked.

As long as stats are clearly inspectable, reasonably anonymized, and are opt-out, I'd be fine with sending them.

Re:Telemetry in the Go Toolchain by marginalia_nu on Wednesday February 08, 2023 @03:40PM (#34714696)
The features that are like fire extinguishers are the ones most likely to be unjustly removed with the rationale of looking at telemetry.

See for example Mozilla's bizarre decision to remove the ability to change the override the character encoding of a webpage with some half-baked detector.

Re:Telemetry in the Go Toolchain by philosopher1234 on Wednesday February 08, 2023 @07:53PM (#34718169)
We cant solve this problem by carving out our eyes. We must adapt to having data about the world.
Re:Telemetry in the Go Toolchain by stonogo on Wednesday February 08, 2023 @09:09PM (#34718862)
Nobody wants you to carve out your eyes. They want you to stop shoving eyes into everything you touch.

Meanwhile, I'll decide what I must or must not adapt to, thank you.

Re:Telemetry in the Go Toolchain by philosopher1234 on Thursday February 09, 2023 @12:19AM (#34720068)
Read more carefully
Re:Telemetry in the Go Toolchain by stonogo on Thursday February 09, 2023 @02:49AM (#34720936)
Write more clearly
Re:Telemetry in the Go Toolchain by philosopher1234 on Thursday February 09, 2023 @11:03AM (#34726024)
Wouldn’t have mattered if I did
Re:Telemetry in the Go Toolchain by simoncion on Thursday February 09, 2023 @05:29AM (#34722000)
If folks were using just their eyes, they'd be calling users in to watch them interact with the software, calling users up to perform user surveys, doing all sorts of live-user testing.

But, that costs money: folks don't generally like to do surveys for free, don't like to come to your site (or have you come to theirs) for free, don't like to participate in tests for free... and the caliber of person who can design and orchestrate all that is _notably_ expensive.

So, companies have discovered that it's far, far cheaper to shove remote eyes into their products. No need to schedule user observation sessions, pay for transportation and participation, have to bother with hiring people who are capable of coming up with a good set of questions and scenarios to walk through.

Re:Telemetry in the Go Toolchain by pmontra on Monday February 13, 2023 @08:24AM (#34773055)
Except that those remote eyes don't look at what a person would look at. They know if a feature was used but they cannot see the frustration on the face of a user when some feature is missing or does not work as expected or is hard to find.
Re:Telemetry in the Go Toolchain by bitwize on Wednesday February 08, 2023 @04:15PM (#34715286)
There are essential, fire-extinguisher-like features. The canonical example is the joke about backup software: if it were developed according to today's standard of telemetry-driven engagement analytics, the restore from backup functionality would be removed because it's used so infrequently.

This actually happens sometimes: when developing the demo ".kkrieger", a first-person 3D shooter in 96 KiB, demogroup theprodukkt tried to shrink it down to get it under the 96 KiB wire. One of the tricks they used was using a profiler to identify code sections that were never reached and could be removed. One of the sections they removed was the handler for the up arrow key in the main menu, simply because the test player never pressed up in the menu.

If you think that Google or another large software organization won't misuse telemetry by cutting or neglecting important but infrequently used functionality to hit some KPI... have you ever worked in a large software organization?

All stats can be deanonymized. The more data you make available, the more you identify yourself. I do not need software I use stealthily tying up bandwidth by "phoning home" with data about me. It is simultaneously betrayal and resource theft. If I wanted to contribute to the improvement of the software, I'd file a bug report.

Re:Telemetry in the Go Toolchain by nine_k on Wednesday February 08, 2023 @05:36PM (#34716538)
I think your view of the ways usage stats are used is a bit simplistic. Not everyone remove "underused" features without giving some consideration, even in big corporations.

But since you clearly don't like telemetry, you should have a way to reliably switch it off. Here we are on the same page: there must be a well-documented and easy way to switch any telemetry off.

Re:Telemetry in the Go Toolchain by bitwize on Wednesday February 08, 2023 @06:19PM (#34717103)
If telemetry is on by default, the vendor obviously wants you to have telemetry on. They are incentivized to make switching it off as difficult as possible, and even pull tricks like turning it back on after a delay of 7 or 30 days or so.

Telemetry should be opt-in, if it's provided at all.

Re:Telemetry in the Go Toolchain by BonoboIO on Wednesday February 08, 2023 @05:27PM (#34716408)
„ Most software features are not like fire extinguishers. „

Amen

Sometimes some pretty niche feature, that is used by one individual that is loud, gets more attention than a feature, used by thousands or millions that are just silent users.

Re:Telemetry in the Go Toolchain by BonoboIO on Wednesday February 08, 2023 @06:59PM (#34717614)
Really, a downvote …