Asking nicely doesn't work in pratice because people willing to answer the question is not a randomly sampled group of people and usually highly biased. You can check how polling works and why it doesn't work.

The problem is always how to randomly sample people and AFAIK there is no good way except randomly sample people (a.k.a telemetry). You can argue that developers shouldn't care that much about their software are being used and whether they are performing as expected, but asking nicely really is not a solution.

> Asking nicely doesn't work in pratice

LLVM, GCC, rust, zig, D, etc don't have telementry and they seem to be doing fine. What makes go special? They aren't trying to be more efficient than rust. It isn't more widely used than GCC. I would argue they don't need this telemetry at all.

No one claimed it's essential to have telemetry or that a language literally can't do without it. After all, Go is doing fine without telemetry too.

But that doesn't mean it's not useful, for Go but also gcc, or Zig, or D, or any other language.

A long list of use cases is given here: https://research.swtch.com/telemetry-uses

Useful doesn't mean it shoudl be done. There are a ton of useful thing for me I don't do because it's not right to do them.

Apparently as soon as a company becomes big enough, the line between convenience and ethical becomes very blurry.

> Useful doesn't mean it should be done.

I never said anything about what should be done. I'm just saying it is useful, as the previous commenter said it's not.

When did I say it isn't useful? I said they didn't need it. They also don't need to install a kernel module that records every keystroke and memory access, but it would certainly be useful data.

Have you asked core developers of those languages how they feel about this? I'd bet that you'd have near unanimous agreement that this class of information would be useful for informing support decisions.

Where I'd expect difference is precisely over the question of whether it's worth the cost & trust issues. That makes me wonder whether there might be some middle-ground here where some third-party like the Linux Foundation could run a telemetry service which is highly public in both its design and collected data, and that would be enough that many people would be comfortable with this kind of service without Google's business reputation entering into the equation.

> useful for informing support decisions

Useful does not equal ethical or legal. A lot of things would be useful to me.

Please enumerate. Go on.

Hence the second paragraph in the comment you're replying to. There's no question that this could be implemented legally but the ethical considerations are legitimate and warrant extensive discussion for an open source community. I think that the Go proposal has been derailed by a lot of hot takes based on scenarios far in excess of what's actually described, but I also think it's quite likely that many languages' core developers would look at that and agree that they don't want that information badly enough to spend time trying to negotiate those hazards.

What if your plumber installed a camera in your bathroom for informed support decisions? Sure would be useful for him to see where you put those "biodegradable" baby wipes after the deed.

Do you really think that's anything like the scenario being proposed?

Here's what I think would be a closer analogy to what's actually being proposed: the water company installs a smart meter allowing them to read usage data more frequently. They randomly sample 2% of customers' data and prepare a histogram showing the top daily usage hours and total usage over a monthly period.

The water company already has a connection to you (the pipe) and the smart meater measures that connection. The smart meter does not collect information how you use that water inside your house. This analogy would match a website collecting anonymous usage statistics for connections but would not match an otherwise offline program making connections to collect data.

Yes, that’s why I think it’s a close fit for what Russ proposed rather than the more outlandish conspiracies circulating in social media.

A more apt analogy would be a sensor which anonymously reports if a pipe starts leaking so the manufacturer has a sense of their failure rate.

I always wondered how much intelligence they gather from the Go playground.

Probably the data is too wild to make any use of, but then all the folks who have a clue using the playground as a kind of proof-of-concept play area... would be great for analysis.

Anyways, I'm not sure how well it would work, regardless of if it should be a thing at all. The majority of stuff I do, and the folks at work, we all have build pipelines that control internet access, use cache systems, etc... so I'm not sure it even matters.

I was actually wondering how much this would affect many build pipelines — if you're doing your builds in a container, the 7 day threshold would never even be reached unless you enabled caching for that file.

The main thing I think this would do is help them flesh out the non-mainstream configurations. For example, I've used Go on AIX but I'd be surprised if bug reports on that platform didn't hit basically every maintainer by surprise since statistically nobody tests on it regularly or even has access to do so. I'd think most of the value of systems like this would be finding out that the change they tested exhaustively on everything Google normally uses is also going to break 100% of some niche community.

FWIW, there is a proposal to add telemetry to LLVM [0] and Rust used to have telemetry [1], both off by default. Some things in the node.js world have telemetry enabled by default, like Next.js [3].

Some people are posting here as if this is already decided -- AFAICT, that's not the case. It's not even a formal proposal yet, and the stated intent was to start a conversation around something concrete. (For context, this is standard for how the Go project approaches large topics, including for example I think there were something like ~8 very detailed generics design drafts from the core Go team over ~10 years).

It sounds like the Go team is going to take some time to look into some of the alternative approaches suggested in the feedback collected so far.

In any event, this is obviously a topic people are very passionate about, especially opt-in vs. opt-out, but I guess I would suggest not giving up hope quite yet.

[0] https://discourse.llvm.org/t/rfc-lldb-telemetry-metrics/6458...

[1] https://github.com/rust-lang/rustup/issues/341

[2] https://nextjs.org/telemetry

C# ( net core ) has telemetry, as well as Java so it's not that it does exists in other very popular languages.

https://www.oracle.com/java/technologies/javase/terms-java-u...

https://learn.microsoft.com/en-us/dotnet/core/tools/telemetr...

Also the idea of GCC is more used than Go and GCC does not have telemetry so GO doesn't need it is completly unrelated.

Most dev working on core tools want telemetry to improve those tools, it's invaluable data to make the right decisions.

> C# ( net core ) has telemetry, as well as Java

Which Java distribution? Maybe the Oracle JDK/JRE? I highly doubt the Oracle JDK/JRE has serious usage numbers because of it's (quite commercial) license.

> dotnet, vscode (Microsoft), golang (Google), Oracle JDK/JRE, Nextjs (Vercel), ...

I wonder if there's some trend where corporate backed, large "open source/source available" projects being more open to telemetry compared to more "open community" projects.

For non-corporate projects, the general thought process might be it's fine to leave efficiency gains and improvements on the table if it means violating some foundational principles. While in a business setting, those efficiency gains could be very tempting since it can translate to more money, market share or promotions and that way of thinking gets applied to the open source projects as well.

I think your analysis is correct, and would also add that corporate projects probably have both the culture of monitoring (they use it for everything else) and the expectation that they have to deadlines so they prioritize the idea of being able to ship something with the confidence that it either won't break anyone or they'll be able to know & react quickly if it does. Open source didn't used to have that same focus since it was rarely someone's job and there was more understanding that time was in short supply (“if it breaks, let me know or send a patch”).

It probably also helps there's more easy access to infrastructure for all of this: servers and sysadmins. Ignoring everything else, that's much harder for your more classic volunteer effort open source project: servers cost money, and maintaining them costs time (usually better spent on actually writing code and such).

> GCC is more used than Go and GCC does not have telemetry so GO doesn't need it is completly unrelated.

I mentioned that because an argument could be made that they're the most popular, or they're trying to be the most efficient language and that's why they might "need" this telemetry data.

> Most dev working on core tools want telemetry

Yet somehow, almost all open source, popular, efficient tools don't have telemetry. Like the ones I listed + python + Linux + coreutils + bash.

Of course they want this. But there are mountains of successful projects that work very well without it, and every one serves as evidence that this Isn't as important as they let on. We shouldn't subject our personal computers to google tracking because they want more tracking data.

In case anyone wants to mention that it's Google's language and they can do what they want with it, this is a perfect example of why people aren't happy with Google's control of the language

> What makes go special?

it's owned by an ad company

> Asking nicely doesn't work in pratice

Well, to bad. Because if you don't, you've built a piece of malware, by definition. You need to ask, live with bias, and draw your conclusions accordingly.

I'm confident that you're not jumping from "asking nicely is not a solution" to "using force or deception is fine."

It's just that, if the argument in my last paragraph did become widely accepted, it would lead to a pretty brutal society. (And, of course, it is becoming pretty widely accepted where software is concerned.)

While true in general, it is really not that necessary to have a representative sample for checking whether your software works. With large enough sample size a convenience sample will find almost all of the bugs that telemetry would have found.

My code is on GitHub as is a large number of other peoples. They can try to compile it themselves with all the profiling and probes they want, on their own machines.

That's not the problem they talking about. It would be useful in some cases (and I'd be surprised if they haven't already considered experiments along those lines) but what you have on GitHub doesn't easily tell them about your build or runtime environments.

Here's a long list of questions Russ Cox was interested in - notice how many of them would really want something like “what is your CI configuration?”, and consider that while they could sample GitHub Actions configuration that doesn't tell them anything about the much more varied set of projects who don't use that service:

https://research.swtch.com/telemetry-uses

That's a very forced, contrived, un-impressive list. It reads like someone told them "hey, we want telemetry, please make up a list of uses that will sound plausible to programmers to justify it. Be sure to use big words like histogram a lot to shut down objections by sounding real smart."

Most of these things are in the category of bug prioritization. Why don't you... you know... talk to users?

If you want some statistics (oh sorry, histograms), maybe pull down the gigabytes and gigabytes of Go code residing on public repositories and places like GitHub and analyze all that. That'll tell you a ton about how people structure their projects, what architectures they target, etc.

Biased sampling only matters if the bias has some relevance to what you’re investigating. There is zero reason for software telemetry to care about privacy bias.

The bigger question is why do they suddenly need this to such a degree that they need to do a backhand tactic to get the data? Most other programming languages have no such data and have survived and thrived for years, decades even before Go existed.

It seems like it’s only a proposal at this point?

Generics were also controversial and it took multiple iterations and years to get them done.

I wouldn’t call it “suddenly”.

Also, why do they need the data “suddenly”? What do you suggest?

It's neither sudden nor backhanded — if either of those characterizations were accurate, we'd have learned about it from something shipping. Announcing it publicly before implementing anything and having a number of privacy protections built-in suggests neither is a fair description.

Reading through the use-cases at https://research.swtch.com/telemetry-uses one thing to consider is that these are the kinds of things people care about as a project matures — once you have a large number of people depending on your project, you have to worry more about unintended consequences as you evolve and since the user community has grown larger and the project more stable it's also harder to know whether the people you hear from regularly are representative. When a project is small or mostly used by enthusiasts you can be more confident that your understanding of what people are doing is relatively accurate but that just isn't true over time. For example, how many millions of Java developers never contact Oracle or the OpenJDK developers throughout their entire careers?

The same argument could be made by many other entities include government bureaucrats and LEOs. Justifying data collection with "it makes my job easier" is a very slippery slope IMO.

By using telemetry you are losing the statistics from people who block it, i.e., the most tech-savy users.

This may be by design, as it punishes those who dare to value their privacy.

eg, some options:

a) No telemetry. All users of the Go language get equal treatment/consideration by the developers ("best guess").

b) With telemetry. The users who opt out are thereby excluded from consideration ("punished").

It seems like over time, option b) would lead to problems for the Go language as the developers base their decisions on incomplete information, whilst believing that information is somehow more complete than option a).

I'm not entirely sure how biased the data would be; there needs to be a correlation between "frobs with telemetry" (whether it's opt-in or opt-out doesn't even matter) and any other metric. I'm not so sure there is one, although perhaps there could be – we need telemetry to demonstrate that :-)

Questions about bias in reported data are legitimate and a genuine topic of concern, but this:

> This may be by design, as it punishes those who dare to value their privacy.

Is just veering off in the conspiratorial.

> Is just veering off in the conspiratorial.

While you may be right, the problem is an obvious one but doesn't seem to be considered.

On the "conspiratorial" front, punishing people who value their privacy seems like it'd be strategically useful to Google, an ad company who has demonstrated their leaders to sometimes be intelligent but ethically challenged people. ;)

Isn't it contradictory to "care" about how privacy objectors use Go, while denigrating those people by suppressing their voices?

> Asking nicely doesn't work in pratice

Who cares?! It's about respect for individuals and respect for their privacy. Any kind of telemetry should be opt-in no matter how you spin the PR.

You seem to have a problem understanding the concept of consent.

Allow me to help you. You ask first. If the answer is no, you don't do it. If you cannot master this skill, or refuse to, you are embarking on becoming one of the most problematic demographics on Earth.

There is no excuse for not asking, and honoring the answer. Sending data is not free, and I guarantee your telemetry is not that important, neither is your vaunted sampling. All you want is to know who your Users are, and the fact is, that is fundamentally gated behind them wanting to even bother telling you.

It’s a simple value proposition: want priorities for your problems and use cases? Turn on the telemetry.

Just like answering polls makes more of a difference than voting.

The world was already there, your bubble just burst. In recent decades tech companies and their programmers have done everything they can to collect data about users. Are you now surprised that the same companies apply the same practices for technical users of their software?

The solution, at least for me, is to remove that software from my life. Personally I vowed a long time ago to never work for a company whose business is to track people. It's no big deal for me, but I understand at least some may have hard decisions to make.

Being able to monitor the system is one of the big reasons to move to web based applications (another being e.g. ease of deployment).

If monitoring of the application isn't possible (And by possible I mean reliably statistically so opt-out not opt in) then that will drive the shift to centralized/web based applications even faster.

Ironically, those who are most wary of telemetry tend to be the same people who also appreciate being able to run local software over web based.

Not sure what your point is. With regards to the last sentence, I don't see the irony. People who don't like being tracked like local software, yes.

The irony I mean that the tracking will be there regardless, the suppliers of software will use "we can't monitor the app sufficiently at end users unless we centralize it" as the argument of not making client side software at all.

That's an overarching claim.

I write some paid local-only software, and I have no trouble not tracking users or usage. For example, here's the entire privacy policy for one of them:

> App does not collect any data.

> App works completely on-device. Messages do not leave your iPhone and are not shared with other apps.

> The essence of this privacy policy will not change.

--

For another software, the privacy policy includes details such as:

> App never uses the Internet, except to check for updates.

> Automatic update checks can be turned off in the application's preferences. Requests to check for and to download updates happen only through the domain domain.example. The updates server does not store or forward potentially identifying information, such as request IP addresses or user agent strings.

> License keys, obtained upon purchasing App, do not contain personal information, such as names or email addresses of license holders. License keys are validated locally; App does not use the Internet to validate license keys.

Well, we'll just have to make our own then, won't we?

Genuine question: How is that ironic? I'm simply not following, which happens to me a lot.

I mean that as a group, peope who crusade against monitoring of client software on end users' machines, will in the long run be accelerating the shift towards more software being web based.

If I'm deciding between a desktop and a web client for my software, monitoring is a concern. If I suspect I'll either a) not get enough opt-ins for montoring or b) end up in a publicity shitstorm if I use opt-out, then I'll just opt to run the software on my server instead.

The amount of monitoring and tracking that is done on server side software is obivoulsy going to be a LOT more than what "anonymous usage data" would be for the client version. And that's a bit ironic (that it might have the opposite effect).

Obviously scenario isn't about this software specifically. A compiler isn't going to be "web based" tomorrow. It was more a comment about the software landscape and the privacy debate in general.

Microsoft has been pumping out proprietary crapware for decades, while those of us who cared about freedom and security have continued to ignore it, despite it even being pop culture at times.

The lack of basic ethics on this topic is a great example of what people mean when they assert that software engineering isn't real engineering. Software should represent the interests of its prospective users, period. Inserting anti-features that directly go against the interest of the user to benefit the author is a violation of trust, and is blatantly unethical.

But sure, keep convincing yourself that if users don't like you violating them a bit, it's just naturally pragmatic to abuse them even worse. Either way whatever you create can't be considered trustworthy.

As for the original topic, the real problem is that it would be a rug pull - classic Google. If this were a new language built with surveillance in the compiler, nobody would use it. But if the maintainers decide to add in hostile features after the language has gained wide adoption, it will take a lot of churn to sort out, fork, sandbox, etc.

Or by paying for it. Google is a powerful & wealthy company and Go is under Google's direct control.

Paying how? People tick a box saying "If you tick this, we'll report how many cache misses your compilation passes did, and in return you'll get a gift card after 10k compilations"?

Why give away a gift card when they can use the data collected from your computer via the compiler to sell the opportunity to sell you a targeted gift card?

Not sure if serious, we are talking about data collection with zero chance of any practical targeted use.

The discussion would be a wholly different one if the data could be used for anything to e.g. sell, target ads or even deduce how/whether an individual has even used the application.

The data can obviously be used for things or google wouldn't pay for servers to collect it?

You sounded like you meant it could be used to target individuals, and not just draw statistical conclusions about the software. Those are quite different things. Every company that does "anonymous usage statistics" (which is basically the norm these days at least for larger apps) obviously have some use for the data or as you say they wouldn't be paying for the functionality.

I am serious that this will start as anonymous and sampled as stated but I am too cynical to think it will stay that way.

or 'tick this box or you're not allowed to access X, Y and Z parts of the tooling".

There's a big difference from functionality not existing to do this, and functionality existing that can change at any time. rsc's attitude about this comes with an air of "I want to alter the deal. Pray I don't alter it further"

> 'tick this box or you're not allowed to access X, Y and Z parts of the tooling'

That would be asshat-y but as long as no PII is transmitted, it would stop short of being a legal issue at least.

This is the thing though I think: people don't like to give up control (in this case, about what their software does). It's not about privacy, it's about control.

I'm sure that will not be gamed at all and that there is zero possibility for this to go wrong.

> why would a compiler need internet access

It doesn't. It compiles without internet access. If it has internet access it can also send telemetry data. If the compiler refused to work in all cases where it couldn't send telemetry data, I'd see the point...

The go toolchain requires internet access to download imported modules unless you aggressively vendor.

Sorry I meant "even when blocking the telemetry calls"

Yes. It is well past time to start using application firewalls (like OpenSnitch) or VM based OSes like Qubes.

https://wiki.debian.org/PrivacyIssues https://www.qubes-os.org/

OpenSnitch won't help if the dependencies you pull in come through the same server as where the tool uploads the telemetry to. Go works with a Google proxy server for this purpose by default.

You can disable the Go proxy and go for a more direct source fetch with an env variable, but in the same way you can also disable the telemetry.

Yeah, I think the only way around that would be for OpenSnitch to block all http/https requests and have a system-wide proxy that sees all the URLs, asks the user, does the approved requests and returns the results.

Somehow we managed to create very high quality software before all this telemetry being larded all over every single thing.

Languages like Go have enormous test code bases with unit tests to ensure that every corner of the language is compiled and executes correctly. We don't need gobs of telemetry to phone home about how people are using a compiler.

There's no guarantee that for some tests that anyone is actually using the edge case, and there's never 100% coverage.

The tests are the map, the actual use of the language is the terrain. The map is assembled largely by blindly guessing what the terrain is likely to look like.

That isn't very scientific.

The result is a situation where users will scream at the language authors if they break something, but they'll refuse to share any information about what they're actually doing with the language, which is perverse.

So instead we get a panopticon where everything we do is watched and can be used to sell stuff to us, profile us, and train AI models using our data so someone else can monetize it. Then there's the security nightmare this represents where secrets can leak via telemetry.

That's not a good trade.

> Do i really have to treat every program as spyware and restrict its internet access ?

My feeling about Google is that they do not respect privacy, no matter the product. Invasion of privacy and tracking their customers are behaviors deeply ingrained in Google DNA.

Doing so for all other products has been incredibly profitable, so why not Go? It’s not like Go the language has a history of respecting the PL community (the resolution of Google stepping on the Go name was Google three their weight around and told the Go! author to pound sand), so why should anyone expect they respect their users?

I have said this before... this is in Google's culture and we must push back on it no matter what. It's basically the boiling pot and the frog.

>By asking nicely. Not by siphoning data out of my system.

THIS !!!

I suspect like many people, I feel that if they had said "We're going to introduce this ... but don't worry, it will be opt-in", then they would have avoided 99.99999999999999% of the negativity.

Instead, by making it opt-out, they are basically doing a classic Google shit move.

And that's before we start getting into questions about GDPR and opt-out.

> Do i really have to treat every program as spyware and restrict its internet access ?

If you're using an application from Google or Microsoft or any big tech company for that matter: yes.

> And anyway, why would a compiler need internet access ?

The go build tool will fetch dependencies for you. If you don't use dependencies or use a local network proxy instead of the Google one you can safely disable WAN access, but otherwise you'll end up with a broken tool.

If you wish, you can ensure that the compilation step in particular doesn't need/use the network, by first fetching the dependencies:

  someproject% go get ./...

Next disable network access for the go command, or for the whole system.

Then compile:

  someproject% go build

That's true. However, because of the design of go tooling, you still call the same command line method to get the tool to compile your code.

This leads to the compiler having access to the internet and possibly exercising its access. In a perfect world of separated concerns the compiler binary wouldn't have any networking code at all, necessitating a go-get before calling go-build.

I understand why the Golang team decided to go with this approach, but it does have side effects that people coming from different tooling (say, C) wouldn't expect.

Sounds like a great reason to adopt Nix! Run each tool in a sandbox appropriate to what it's supposed to do, and store the entirety of the inputs and outputs so they can be audited if the need arises.

Opt-out should never be the default.