News for hackers. Stuff that matters.
Re:Rust 1.43 by digikata on Thursday April 23, 2020 @11:53AM (#22957299)
The added cargo env variable pointing to the path of generated binaries is a nice quality of life fix. I think that was one of the first annoyances I ran into with cargo and binaries. It was minor but was like a little splinter that stood out from cargo's otherwise nicely integrated build and test capabilities.
Re:Rust 1.43 by weiming on Thursday April 23, 2020 @12:34PM (#22957801)
I've been using https://docs.rs/assert_cmd/ for this. (Helps spawn the binary and assert output.)
Re:Rust 1.43 by epage on Thursday April 23, 2020 @02:12PM (#22959033)
Thamks for the call out! I'm excited by the new env var and need to consider its impact on `assert_cmd`.
Re:Rust 1.43 by bsder on Thursday April 23, 2020 @03:55PM (#22960425)
The real problem though is that cargo doesn't play nice as part of system where cargo doesn't own everything.

Cargo needs a lot more ways to be interrogated about the artifacts it produces and control over how they are produced.

Re:Rust 1.43 by lilyball on Thursday April 23, 2020 @04:17PM (#22960732)
Exactly this. It would be really great if for example when building Rust in Nix, Nix could own each of the artifacts, in order to share artifacts between tools that use the same revisions of dependencies, as well as to avoid rebuilding artifacts unnecessarily on every change to a derivation.
Re:Rust 1.43 by digikata on Friday April 24, 2020 @12:49PM (#22970160)
How does Nix handle areas with extensive package ecosystems? Python, Node et al? It seems like every distro/super packager has different compromises for this question.
Re:Rust 1.43 by lilyball on Friday April 24, 2020 @02:09PM (#22971218)
I haven't investigated Python, but for Node it uses a tool node2nix that fetches the metadata for the entire dependency tree and constructs a Nix expression for each one, such that each expression represents the source for that specific dependency. It then takes the top-level package and makes a call to `nodeEnv.buildNodePackage`, which contains the source for that top-level package and declares the entire recursive dependency tree. buildeNodePackage itself rebuilds the entire node_modules folder from the declared dependencies and reconstructs the lockfile. And then it asks npm to rebuild the package.

The end result is this bypasses npm's downloading and managing of node_modules, does it all in a fully declarative fashion, and then hands control over to npm to build the package. Also, each dependency's source is cached independently by Nix, so if multiple packages use the exact same dependency, it doesn't have to be redownloaded (unlike Rust where the entire dependency tree is vendored separately per package).

Re:Rust 1.43 by steveklabnik on Friday April 24, 2020 @04:44PM (#22972726)
> (unlike Rust where the entire dependency tree is vendored separately per package).

To be clear, Cargo downloads the source globally. The compiled output is per-project, however.

Re:Rust 1.43 by lilyball on Tuesday April 28, 2020 @05:04PM (#23012001)
Nix uses `cargo vendor` to literally vendor the source of all dependencies as a separate derivation. Maybe cargo keeps a global cache of the source as well when doing `cargo vendor`, I don't know, but Nix does construct a derivation that contains just the vendored dependency sources and then provides that as input when building the full crate.
Re:Rust 1.43 by jeffdavis on Thursday April 23, 2020 @12:27PM (#22957712)
I'm curious why there's no similar variable pointing to a built library (e.g. a cdylib).
Re:Rust 1.43 by steveklabnik on Thursday April 23, 2020 @12:31PM (#22957768)
What's the use case?
Re:Rust 1.43 by jeffdavis on Thursday April 23, 2020 @01:49PM (#22958697)
I feel like I've been pushing my use case plenty already, so please take this as an answer to your question rather than one more push ;-)

I'd like to be able to create plugin-style cdylibs that get loaded into a host program. For instance, PostgreSQL extensions are shared libraries (typically written in C) that need to be loaded into a running instance of postgres for integration testing. I am trying to make it reasonable to write such extensions in rust (https://github.com/jeff-davis/postgres-extension.rs).

The environment variable to find the shared library would solve one problem, but there are still a couple more right behind it:

* Running "cargo test" tries to build a binary that links to the library to run the unit tests, but this will fail because there are unresolved symbols in the .so file (symbols that call APIs in the host program, in this case PostgreSQL functions like palloc()). In other words, the test binary is useless because it can't load the plugin.

* Linux seems fine creating a .so with unresolved symbols, but Mac and Windows need special linker flags. I can document that the linker flags are needed by consumers of my crate, but it would be nice if there was a better/supported way to create plugin-style shared libraries. I tried filing a bug with PR (https://github.com/rust-lang/rust/pull/66204) it was determined that it needed to go through the RFC process.

The simplest way to see the problem is to look at this crate: https://github.com/jeff-davis/unresolved

My problems don't end there, unfortunately. For my crate to really work, I also need to solve the problem of setjmp/longjmp at the edges between C and Rust (https://github.com/rust-lang/rfcs/issues/2625).

Re:Rust 1.43 by steveklabnik on Thursday April 23, 2020 @01:55PM (#22958770)
It's all good; I was asking because I legit didn't know, and couldn't think of anything off the top of my head. I don't work on Cargo, so I'm not as in-tune with its development.

So, I am still not 100% sure that I understand, mostly because each package can only have one library, so this would only work if you wanted one plugin. I guess maybe if they were all in a workspace?

Re:Rust 1.43 by jeffdavis on Thursday April 23, 2020 @02:14PM (#22959056)
Typically one postgres extension would have only one plugin library. So, the idea is that one crate would produce one plugin. I guess it's possible to have more than one plugin per extension, but it never occurred to me and I haven't seen that before.

Having the location of the plugin might also allow "cargo install" to work.