News for hackers. Stuff that matters.
Re:Rust Without Crates.io by josephg on Tuesday November 14, 2023 @07:53AM (#38262699)
I suppose I could manually traverse my whole dependency tree myself and install all those packages myself. Set up cmake and pkg-config for all of them. If anyone else wants to try out my software, they’d have to manually look up all the dependencies in my github (?) and install those too. And hope the versions all line up nicely - which is a big if during development. Maybe I could script all of that dependency tracing - if only there were some standard software to do that. Guess not.

If C/C++ is all you know, you’re used to how painful all of that is. But what you’re proposing is a significantly worse experience in every way compared to what I can do with cargo or npm today. None of the claimed benefits seem remotely worth the cost of wading back into C dependency management hell. And cargo and npm show just how unnecessary all that pain is in C!

Just make good tooling. We can have nice things. I stopped using C not because I decided to write assembler. I stopped when better languages came along. You want me to stop using cargo? Nobody wants to go back to manual dependency management. Make something better.

Re:Rust Without Crates.io by mid-kid on Tuesday November 14, 2023 @08:00AM (#38262771)
If you're smart about it you're not gonna have a very deep or wide dependency tree for your own deps. If you need more than maybe two of your own deps, maybe you need to reconsider how you structure your libraries, or you can join multiple libraries into the same project/build system umbrella (e.g. glib/gobject or the whole of QtBase)
Re:Rust Without Crates.io by chalsprhebaodu on Tuesday November 14, 2023 @08:11AM (#38262865)
Why is this inherently better than better tooling for dependency management?
Re:Rust Without Crates.io by josephg on Tuesday November 14, 2023 @08:47AM (#38263223)
Why would I want to join unrelated modules together into a kitchen sink library? The only benefit of that is to work around bad dependency management systems that make dependencies a hassle to work with. But otherwise, they’re a bad idea. Packages should do one thing and do it well. If I want a good random number library, I don’t want to import bob’s junk draw of random code. I want a small, reusable random number library. I want one that I chose. And my users want one they chose. And if that choice is bad, I want to be able to replace it without throwing out half my ecosystem or breaking downstream compatibility.

QtBase is a symptom of C++’s bad package management. 8 unrelated utilities should be in 8 small packages that users can pick and choose based on their needs.

Re:Rust Without Crates.io by mid-kid on Tuesday November 14, 2023 @12:43PM (#38266681)
It's less a workaround around bad dependency management, and more about the fact that it's simply easier to reason about a smaller set of dependencies regardless of the ecosystem. This includes both vetting, ensuring version compatibility, checking license compatibility and reducing the SBOM, as well as governance over the project and integration between different modules.

Too often have I seen the equivalents of frameworks in other languages being split over sometimes hundreds of packages, that don't always make it clear that they're to be updated in tandem, what their exact relationship is, and that the same organization manages all of them.

As for QtBase, it's a superproject, but that doesn't mean that you can't use its individual modules separately, and depending on the distro (e.g. debian) install them separately as well. A singular project installing multiple related libraries makes a lot of sense.

Re:Rust Without Crates.io by josephg on Tuesday November 14, 2023 @03:30PM (#38269283)
> This includes both vetting, ensuring version compatibility, checking license compatibility and reducing the SBOM, as well as governance over the project and integration between different modules.

Auditing large code bases takes disproportionately longer than auditing small code bases. So I don’t think that’s a win. License compatibility is trivial to check. SBOM is strictly larger if I pull in a kitchen sink package because I’m probably not using most of the stuff inside. Better to just pick out the components I want.

The one thing I’ll grant you is that shared ownership and visibility means it’s less likely that one rogue person will sneak ransomware into the dependencies.

Personally I’d love a capability based package manager that lets me pass a package exactly the capabilities it needs to do what I downloaded it for. Why does every package need access to my files when I install or run it? That’s ridiculous. Totally unnecessary and a massive security risk. We could solve that C++ style by hobbling package managers and using fewer, jumbo packages. But that doesn’t solve the root problem. I want a package system in a language which lets me pass capabilities to each library I use, when I need to use them. Eg “Please open your web server on the port associated with this capability token.” This needs language support too but it’d be so much better from a security point of view than anything that came before.

Re:Rust Without Crates.io by Ar-Curunir on Tuesday November 14, 2023 @08:49AM (#38263238)
This is an example where your tooling limits your solution space. There’s nothing inherently wrong with a deep or wide dep tree.
Re:Rust Without Crates.io by SkiFire13 on Tuesday November 14, 2023 @09:00AM (#38263364)
> or you can join multiple libraries into the same project/build system umbrella (e.g. glib/gobject or the whole of QtBase)

This also seems worse than the status quo. So instead of having 3 or 4 dependencies I now only one that combines 20 different ones I don't need.

Re:Rust Without Crates.io by patrick451 on Tuesday November 14, 2023 @09:16AM (#38263538)
> And cargo and npm show just how unnecessary all that pain is in C!

Leftpad would like a word.

Every dependency is a liability. It's another thing to vet, another thing which can disappear. Having few dependencies in c is a feature, not a bug. The approach to dependencies in rust was a major turn off to me. I recall 15 just to get random numbers in the intro tutorial. No thanks.

Re:Rust Without Crates.io by josephg on Tuesday November 14, 2023 @03:17PM (#38269058)
Just in case you don’t know, packages can no longer be pulled from npm by the developer. At least not without emailing someone. And you need a very good reason - like that version is ransomware.

The particular pulled package version identifier ([email protected]) can also never be reused. If you audit 1.0.1, it will never change out from under you.

The leftpad fiasco was hilarious and embarrassing. But it can’t happen again because of changes in npm policy.

Re:Rust Without Crates.io by navi-desu on Tuesday November 14, 2023 @11:29AM (#38265488)
meson actually solves this problem on the build system, their wrap subsystem allows you to use the system provided lib if found, or fetch from a database during the build process if not.

best of both worlds, imo.

Re:Rust Without Crates.io by cozzyd on Tuesday November 14, 2023 @12:18PM (#38266267)
Keeping track of C/C++ dependencies is basically the point of Linux distros...

Cargo's main problem is that it's (mostly) rust only. Same problem that pip/npm have, relegating you to (mostly) single-language silos.

Re:Rust Without Crates.io by humanrebar on Tuesday November 14, 2023 @05:34PM (#38270799)
...except pip/npm often build C or C++ while packages install, at least in some cases.

And Rust has the same issues, at least when shipping or consuming libraries with C APIs is in the picture.

Re:Rust Without Crates.io by cozzyd on Tuesday November 14, 2023 @06:03PM (#38271081)
Yes, sometimes, but it won't resolve C/C++ dependencies in general.
Re:Rust Without Crates.io by josephg on Tuesday November 14, 2023 @05:42PM (#38270872)
> Keeping track of C/C++ dependencies is basically the point of Linux distros...

Judging by the proliferation of docker I don’t think they’re doing a great job of even that. Docker shouldn’t be needed if apt was up to the task. Nixos is a much better design imo.

> Cargo's main problem is that it's (mostly) rust only.

True. But apt’s main problem is that it only works on a few distributions of Linux. Limiting the users of my rust library to rust developers is usually not a problem. Limiting the users of my rust library to the people who use homebrew or arch or something is not fine. And I don’t want to redo all the packaging work on every operating system and major Linux distribution. Life is too short for this make work BS. Especially given it could be automated away anyway. Rust packages already explicitly define all their dependencies. Why can’t apt just generate .dpkgs from my Cargo.toml?

Apt’s other problems are that packages need to be installed globally. And they make a mess of versions. And there’s no feature flags. Run `apt search vim` some time and see how many packages they have! Or the same for llvm. It’s a dog’s breakfast.

The time that cargo’s rust focus hurts it is with binary packages. It’s not enough for ripgrep to live in cargo since it’s designed to be useful for people who aren’t rust developers. And I don’t have a good answer for how to solve that. I wish I did.

Re:Rust Without Crates.io by cozzyd on Tuesday November 14, 2023 @05:51PM (#38270969)
We'll this also works for rust libraries as they're usually useless for other languages, whereas libraries like gdal, opencv, fftw, etc. can be used by many languages. (Though to be fair, until rust gets a stable ABI it's not going to be a good choice for this sort of thing).

I personally consider docker/podman just completely punting on dependency management, and try to avoid them, but I'm probably in the minority...

Re:Rust Without Crates.io by 15155 on Wednesday November 15, 2023 @01:20PM (#38280263)
Rust has a stable C export ABI.
Re:Rust Without Crates.io by cozzyd on Wednesday November 15, 2023 @03:50PM (#38282274)
Funnily enough, the most notable rust-implemented with C ABI shared library I'm aware of (librsvg) just became available on crates.io a few months ago...