C++ does not have an equivalent, it's completely decentralized which results in more messy situation. As a result you have an open market where different people try to build different tools and approaches for their own problems, then try to get others to use them (similar to what Go had before go modules, we had lot of package managers to chose from at the time).
Instead of a top down decision it's a negotiation between the various actors. But the last thing we need is for the C++ standards committee to standardize a package manager. That would take forever to do, would result in a messy tool that tries to compromise with all the actors in some ways, make it very hard and slow to evolve over time and would likely result in a lot of pain, etc.
That is the role of the ISO C++ committee, is it not? They are the major stakeholders. They would just have to care enough. They cared enough to release C++20, didn't they? It's not like they never get anything done, which seems to be the implication a lot of people make in this discussion.
> Instead of a top down decision it's a negotiation between the various actors.
My understanding is that the various committee members represent the disparate interests of the broader C++ community. I agree it would be very much like a negotiation.
That doesn't mean that it can't be done. This whole thread is discussing things that have been done by the C++ committee: C++20.
> But the last thing we need is for the C++ standards committee to standardize a package manager. That would take forever to do, would result in a messy tool that tries to compromise with all the actors in some ways, make it very hard and slow to evolve over time and would likely result in a lot of pain, etc.
You just summarized my feelings about C++ in general. I would much rather people use Rust or Go or any number of other languages instead of C++, depending on project needs. Such opinions are rarely taken well in threads like this, though, so...
I've been trying to be optimistic and point out that C++ could get package management. If the C++ committee process works well, then the package manager should also end up turning out well.
I'll leave the reader to decide how well they think the long term direction and guidance of the C++ standard has been going and apply that to their feelings of a hypothetical future package manager.
That being said, the C++ standards body (at least under Herb?) has done a decent job modernizing their process to fight some of the gravitational issues they were having. They've formalized deprecation rules & tried to get over disagreements. The design by committee issues haven't gone away though - the mess with coroutines, modules, & concepts is a great example of that. The ISO process of language papers precludes even simple additions to the STL where you not only have to navigate standardese, but also manage the review process (that's why you have to find a champion on the standards body to help guide your review through the rigamarole).
My experience contributing to the Rust standard library by comparison was much easier - put up a drive-by diff adding a new (admittedly minor) API, some minor review comments, done & shipped. The whole process took 1-2 weeks, no standardese, no arguing with a large committee on the exact wording, etc.
this is off course all great until two people working in different parts of the language do things slightly differently. Either works alone, but the whole of the language is inconsistent and hard to learn.
C++ has enough inconsistent parts already and so tries to be careful to make new things consistent with itself as best as possible. Even that has failed despite all the review of people looking for places to make things consistent. It is a hard problem to design a large language.
The whole language team has to sign off on these stabilizations and language additions, which is what keeps up that consistency you're talking about.
Granted, this isn't necessarily everyone's experience in std as the change I implemented is well-worn/adopted by any condition variable implementation. Something more controversial/exploratory may have been pushed off into a crate first. I'm still impressed that it only took ~2 weeks to get [1] reviewed & into unstable. I wasn't even involved in the stabilization work/cosmetic renaming that it took to close out [2] which was driven by a community ask & the std maintainers doing a pass to make things consistent. Rust's velocity seems to be that they can deliver changes a full 1 year faster than C++ can (& likely faster if the community really asks for it). In my book it's largely owing to having 1 compiler & 1 standard library & the latter having a much more streamlined RFC process.
[1] https://github.com/rust-lang/rust/pull/67076 [2] https://github.com/rust-lang/rust/issues/47960
Some people do occasionally use Bazel or other build systems on top of the Go build system for complicated monorepos.
Thinking about how much or how little would be required beyond a common build system in order to get a working package management system is still a valid thing to do.
Keeping a parallel set of instructions or metadata that includes specific URLs and such might work, though. As long as you can skip all that when a system package, filesystem path, or git submodule is more appropriate.
Are you talking about "pkg.go.dev" and the "go get" command? Isn't there some path dependence in the history of events that's not comparable to C++? Consider:
- Go language: created by Google Inc
- "go get" syntax for package download designed and created by Google Inc
- "pkg.go.dev" funded by Google Inc and highlighted on "golang.org" website that's also run by Google Inc.
There is no business entity or institution in the C++ world that's analogous to Google's influence for Go + golang.org + "go get" + pkg.go.dev.
>It would just require the major stakeholders to all _care_ enough to make it happen,
But it's easier to care if there was an influential C++ behemoth that captured everyone's mindshare to move the entire ecosystem forward. C++ has no such "industry leader" that dictates (or heavily influences) technical direction from the top down.
No. I'm not talking about either of those. Your whole comment is, unfortunately, irrelevant.
pkg.go.dev is not a package repo. It's just a place for documentation to be rendered. It renders documentation from third party hosted code, such as on GitHub or elsewhere.
"go get" predates Go Modules, which is the current package management system. The whole original design of "go get" was to simply download code from somewhere on the internet, and place it in the right spot of the $GOPATH. This has nothing to do with a proper versioned package manager like Go Modules.
AFAIK, "go get" was also never really designed for Google's internal use cases. They use a monorepo that was perfectly content with $GOPATH, and all their code was developed in the monorepo to begin with. There was nothing for them to "go get", except for the rare outside dependency that they were embedding into their monorepo, I would imagine. I've never worked for Google, these are just things I hear about.
Go Modules was also not designed for Google. It was designed for the community, based on findings from community developed package managers for Go. Google has no real use for it — again, they use a monorepo.
Nowadays, "go get" can be used with Go modules, but in practice, it feels like it almost never is. Maybe someone would use that command to upgrade an existing dependency, instead of editing the `go.mod` file to change the version there?
So, your comment just shows that you haven't researched this enough. Yes, Go Modules was still guided by Googlers, who were even more in control of the language direction back then than they are now. Yes, change always causes some drama. But, I'm not really here to explain the history of Go package management...
I'm just saying that C++ could have a nice, distributed package management system, it would just require the major stakeholders to all care and work together on it. The ISO C++ language committee is a finite number of people. They are the major stakeholders, as far as the language direction is concerned.
If they didn't have the power to enact major language changes, we wouldn't be here talking about C++20.
The stakeholders for Go were able to develop a package manager that is distributed (an idea compatible with how all C++ code is scattered across the web these days), and that achieved broad adoption, and this was some years after the language went into wide use.
It’s an extremely relevant analogue for C++ to study, if the committee members wanted a package manager badly enough.
> But it's easier to care if there was an influential C++ behemoth that captured everyone's mindshare to move the entire ecosystem forward. C++ has no such "industry leader" that dictates (or heavily influences) technical direction from the top down.
You edited this in while I was replying, but I agree entirely. Getting the committee to agree to a package management solution would be much more difficult than having a single behemoth guide the decision. Does that mean it is impossible and therefore no one could do it? Everyone here talks like it is impossible, but it doesn't really seem to be.
>I'm just saying that C++ could have a nice, distributed package management system, it would just require the major stakeholders to all care and work together on it. The ISO C++ language committee is a finite number of people. They are the major stakeholders, as far as the language direction is concerned.
The ISO C++ committee can't learn from the history of Go modules community acceptance because they don't have the same power as Google. You seem to misunderstand what the C++ committee _is_. Yes, they have representatives from Microsoft/Google/Apple/Intel but the org is designed to review proposals from submitted papers. They are more like an ongoing academic conference rather than a devops team that runs websites.
We seem to be discussing 2 different abstractions of making a "package manager". With your emphasis on Modules, you seem to be only focusing on the tool. To repeat my gp comment, I'm also focusing on the canonical package repository (or index, or discovery engine).
Consider the following sentence from https://proxy.golang.org:
>The Go team is providing the following services run by Google: a module mirror for accelerating Go module downloads, an index for discovering new modules, and a global go.sum database for authenticating module content. >As of Go 1.13, the go command by default downloads and authenticates modules using the Go module mirror and Go checksum database.
You misunderstood my cite of "pkg.go.dev" run by Google Inc but this is the part of your survey that I was referring to:
>The package discovery site pkg.go.dev is new to the list this year and was a top resource for 32% of respondents. Respondents who use pkg.go.dev are more likely to agree they are able to quickly find Go packages / libraries they need: 91% for pkg.go.dev users vs. 82% for everyone else.
The ISO C++ committee is not set up to implement a new website to make the above Go-specific paragraphs be a similar reality for C++ with a search & replace "s/Go/C++/g". Think about _who_ funds and provides paid people to actually run the "proxy.golang.org". It's Google Inc. The C++ committee doesn't have an equivalent situation.
Yes, the C++ committee can receive a proposal for new language syntax such as "std::unique" and after some back & forth commentary and debate, they say "approved" and then it's up to each C++ compiler vendor to then go and independently implement it on their own timeline. In contrast, if someone proposes "C++ should have a package manager", exactly _who_ will implement and maintain the canonical repo mirror? This is not independent lines of work that GCC, Clang, Microsoft, and Intel can do on their own. Even if we hypothetically extend the website "isocpp.org" to actually start hosting the canonical C++ repos instead of just blog posts about syntax proposals, _who_ is paying for it? Again, there is no single entity like Joyent/Mozilla/GoogleInc that raises their hand and says, "We'll set it up". I suppose we could imagine that the major players like MS+Google+Apple all contribute to a shared fund to pay for the repo mirror -- and the salaries for devops to remove malicious uploads -- but notice no other major language package manager Javascript/Rust/Go had to do it that way. So we have that friction of coordinating multiple corporations. Even if that website was set up, many existing C++ library writers (that existed for decades before a C++ package manager) wouldn't bother uploading their code to it. So that's another friction. E.g. Conan is supposedly the current winner of C++ package manager mindshare and ffmpeg is not on it.
I think the disagreement is rooted in how we compare ISO C++ committee vs Google Inc. To me, releasing a C++20 language specification* does not say anything about implementing a canonical repo* so that a command line tool magically works the way people expect.
EDIT to reply: >You can have a package manager without having a discovery tool or a central repo.
This means your conversation is focusing on the tool which isn't the abstraction I'm emphasizing.
>Package discovery tools are not very relevant to the discussion.
It's relevant if the particular person wondering "why C++ doesn't have a package manager?!?" uses a mental of model of how npm and cargo work. They don't have to know if it's github vs gitlab vs somewhere else. The tool just works without thinking about the location. That's what a canonical repo as a default convention for the client tool provides.
Your edit implies that I'm talking about a useless tool that can't do anything in the absence of Google, which simply isn't true.
The Go Modules tooling does not depend on any central resource to work. Google could shut down tomorrow, and nothing would change for existing projects. The go CLI tools would still be able to find, download, and verify the dependencies. I would still be able to add new dependencies, and the tooling would be able to fetch those.
What are you talking about, if not a functional package management system? Google's websites are nice, but they're not required for everything to Just Work.
Anyone in the C++ community could stand those websites up at any time after the package management tooling came into existence. They're not required for the functionality of the package manager.
> It's relevant if the particular person wondering "why C++ doesn't have a package manager?!?" uses a mental of model of how npm and cargo work. They don't have to know if it's github vs gitlab vs somewhere else. The tool just works without thinking about the location. That's what a canonical repo as a default convention for the client tool provides.
Go's CLI tooling literally doesn't provide any way to search for packages at all. You may think it's a requirement, but it's really not! Go requires you to know where the dependency is located, because Go sure doesn't unless you tell it!
It feels like I'm really awful at explaining things.
You're still misunderstanding the level of abstraction I'm emphasizing for what a "package manager" means to many people.
Let's dissect the following command from the Javascript ecosystem:
npm install react
Notice that the end user does not need to know whether React is hosted on GitHub or GitLab or Facebook's own servers. He doesn't even have to do a google search. The npm command just "magically" gets the React library.Exactly _how_ does npm do that? From _where_ does npm fetch? The _how_ & _where_ is what the majority of my comment is about. All your explanations of Go not working that way does not address that mental model at all.
So you have 2 concepts in a "package manager":
(1) npm, the client command line tool
(2) the canonical default repo that npm tool points to -- and it's a virtuous cycle of easy use and trust because almost everybody publishes to it. It has grabbed mindshare.
You keep saying Go doesn't need (2) but I'm saying that doesn't change the fact that many mentally include (2) of what a comprehensive package manager _is_.
And is (2) really that unreasonable? Consider the Go documentation example of "adding a dependency" from https://blog.golang.org/using-go-modules#TOC_3 -- it has this example output:
go: finding rsc.io/quote v1.5.2
go: downloading rsc.io/quote v1.5.2
... exactly _where_ is it downloading the "rsc.io/quote" package if the user does not manually specify Github/Gitlab/PrivateEnterpriseRepo ?It's getting it from https://rsc.io/quote
I don't understand what's confusing about this... it's literally specified in the name of the package.
Russ Cox is hosting his packages at rsc.io, which is a personal domain name he owns. If you visit it with a normal browser, he just kicks you over to pkg.go.dev, because he doesn't want to put in the effort to make a website for your human consumption. He's just hosting some packages there.
I really, really feel like you need to spend some time with Go Modules. You don't really seem to be getting the decentralized nature of it. But it works, and it works well!
In this case, Russ Cox has a meta tag there that tells the Go tool to download it from GitHub: <meta name="go-import" content="rsc.io/quote git https://github.com/rsc/quote">
But there's nothing stopping him from actually exposing a git repo at https://rsc.io/quote, instead of just exposing a redirect.
By telling people to use that package URL, he has the flexibility to change how and where he hosts the package in the future.
Go has proven that a good package manager can work without that feature. You say that feature is something the C++ Committee could never tackle. My whole statement has been "fine, learn from Go!" Instead, you keep harping on this nice-to-have feature and saying it can't be done.
Package management is solvable in a way that suits C++. It seems inevitable that standardized package management will eventually happen for C++.
I understand your perspective now, but I just don't think I agree with it.
Literally no one is required to do any of that. That is the answer. Plain and simple.
> I think the disagreement is rooted in how we compare ISO C++ committee vs Google Inc. To me, releasing a C++20 language specification* does not say anything about implementing a canonical repo* so that a command line tool magically works the way people expect.
But I'm saying that's not how Go works at all. The dependencies are hosted on GitHub, GitLab, or wherever else.
There is no central package repo. There is no "canonical repo".
Package discovery tools are not very relevant to the discussion. You can have a package manager without having a discovery tool or a central repo. Searching GitHub to find a C++ package, then adding that repo as a dependency of your current project seems like it would be entirely reasonable, if C++ had a standard package manager that worked. Some community members might build a website to help you find popular packages... but that discovery tool doesn't interact directly with the packages at all.
proxy.golang.org is a proxy. No one publishes packages to it, and you don't have to use that proxy. You can use no proxy at all, which was the default once upon a time, or your company can host a proxy, or you can potentially find some random third party proxy online. The proxy isn't where packages are hosted -- it's just a means of accelerating downloads, if GitHub were slow, for example.
C++ code is hosted in a myriad of locations. The Go approach is to specify the GitHub repository that you're depending on, and that repo will be cloned by the package manager in your terminal. The `go.sum` file contains hashsums to verify that the dependency you downloaded is untampered with since the last time you fetched it, and those hashes can also be used by any proxy that happens to be used.
Go's package management system is truly distributed. It isn't centralized at all. Yet it still supports SemVer, downloading the correct, exact version of a dependency, checking the integrity of dependencies, recursively collecting dependencies of your dependencies, etc. All the features you would expect out of a package manager.
Unlike Cargo in Rust, someone can delete one of these repos from GitHub and cause a real mess. `go mod vendor` is an option for anyone who prefers to vendor their dependencies.
Google has certainly provided some nice web tooling around the Go Modules system, but none of it is integral to the type of package manager that I'm proposing would suit the C++ dependency model. Go's package manager is very distinct from what you were discussing with Cargo, NPM, and others. It's much more attuned to the problems that C++ faces, and it walked a similar path to what C++ will inevitably have to do.
Source: I build lots of Go modules for fun (well, specifically, I have some automation to do it for me) and notice these things, when I get more failures than expected. http://github.com/vatine/gochecker for anyone wanting to play along from home (the 1.16 release report is on hiatus, as there was a LOT of things that didn;t work smoothly this time).
Go's package manager also came years after the language became widely used, and it is now very widely adopted according to the most recent survey[0].
I think C++ could have a good, unified package management story. It would just require the major stakeholders to all care enough to make it happen, which seems to be the missing piece here.
[0]: https://blog.golang.org/survey2020-results#TOC_8.