The simplest way would be to package the app for guix and you could just run '$ guix environment <name-of-package>' and you would be dropped into an environment with all your dependencies and whatever else the application requires in your path ready for hacking, get your sources and editor and start working.
If you need a vm or similar though I'd translate your example above into a system config where:
- packages include python-2.7 and whatever is in requirements.txt (this may mean you have to package a few things, but again this is usually super easy)
- users and groups are added to the config, as they always are, no extra step necessary.
- exposing ports and networking is available as options for qemu script guix produces to launch the vm.
- CMD ./notify.py: create a "simple" service that can be autostarted by the system on boot.
- filesystem access is also handled by arguments to the qemu script.
As always though there are several paths to Rome, and these are just two of them.
Zeromq and libsodium are already packaged on guix, czmq and zyre looks like they would be simple to package, guix is really quite simple to work with, which I think is the reason so many of the users and devs are running it as our daily drivers, even though it is strictly beta (0.14. I think is the last release).
And pointless, come on - what does that even mean? Does it mean you don't value them? I was quite happy to read about a neat new thing I can use my favorite tool for.
Yes, I know all that. It's neat. I would like to learn more about it.
> The simplest way would be to package the app for guix
I was asking how to package the app for guix, and your response is the simplest way would be to package the app for guix...
> If you need a vm or soimilar though I'd translate your example above into a system config where: - packages include python-2.7 and whatever is in requirements.txt (this may mean you have to package a few things, but again this is usually super easy) - users and groups are added to the config, as they always are, no extra step necessary. - exposing ports and networking is available as options for qemu script guix produces to launch the vm. - CMD ./notify.py: create a "simple" service that can be autostarted by the system on boot. - filesystem access is also handled by arguments to the qemu script.
Yes, I'm sure it is super easy. How do I do it?
Do you know how to use the dockerfile I posted above? You run
docker build -t myapp .
docker run myapp
that's super easy. 9 lines and 2 commands. You can now add docker expert to your resume.> Zeromq and libsodium are already packaged on guix, czmq and zyre looks like they would be simple to package,
Well, I was working on a fork of things, so I would have needed to install my forks.
> guix is really quite simple to work with
I'm sure it is!
> And pointless, come on - what does that even mean? Does it mean you don't value them? I was quite happy to read about a neat new thing I can use my favorite tool for.
You are correct, I don't really value posts saying how cool and easy something is and how much better it is than other solutions, when they don't actually present a complete solution someone can actually use.
I get that it is not other peoples job to teach me how to use something like guix, but do people not understand why things like Docker won?
I'd be happy to provide some examples. Say you want your fork of libsodium:
(define-public my-libsodium
(package
(inherit libsodium) ; now anything not defined in this package will be inherited from libsodium
(source (origin (method url-fetch)
(uri "url-to-your-sources")
(sha256 (base32 "hash"))))
; Add whatever other fields your fork needs.
))
Sure it's slightly more verbose. That's a bit of the cost of having something you can actually rely on, with that degree of hackability.If you actually want help to package these things ask on our mailinglist or IRC, we're happy to help with specifics. But you're basicly complaining that I didn't give you a concrete solution to a problem with several missing details that are important. Docker would not be able to instantiate your python project if it did not know the contents of your requirements.txt.
The thing is docker is huge and bloated; is far from secure, and will probably stay that way for the foreseeable future; has a more or less complete lack of introspection; and is not strictly reproducible (sure, it gets quite far along the way, but it really is not).
Guix on the other hand is rather lightweight, and you have a fair amount of control over how lightweight it should be; builds from source, and has a sort of hotpatching system for security fixes; has introspection and is quite close to bitreproducible.
Sure, docker is _easy_, as long as it works. And I'd argue that because of its complexity and obscurity it is not practically free software.
For instance, (from the openSUSE community which I'm a part of) we have KIWI that provides builds with full introspection on a package level (similar to what you're doing with Guix). If you build the image inside OBS (our build system) then if a dependency of your image is updated then your image will be rebuilt automatically and published in OBS (where it can be further pushed to any Docker/OCI registry you like). The packages are signed, and the image is also "signed" (though it currently signs the image artifact and doesn't use image signing since that is still not standardised). And most packages in openSUSE are bitreproducible (we build everything in OBS).
The above is far and above much better than the current standard in the "official" world of Docker, but unfortunately because OBS has a UI from the early 2000s (which is when it was written) it doesn't get enough attention outside of the communities that use it (and enjoy using it a lot). Everyone wants Dockerfiles even though they cannot provide these features (and you cannot get package manifests of your images without running a package manager in the image, which means you cannot get vulnerability information from the manifest).
[ Though I'm mostly talking about openSUSE here, I also happen to work for SUSE on the containers team. ]
And Guix is one of them, remember? From the article:
> Add -f docker [to your `guix pack` command] and, instead of a tarball, you get an image in the Docker format that you can pass to docker load on any machine where Docker is installed.
:-)
> The above is far and above much better than the current standard in the "official" world of Docker, but unfortunately because OBS has a UI from the early 2000s (which is when it was written) it doesn't get enough attention outside of the communities that use it (and enjoy using it a lot).
This is so true! I've mostly moved on from traditional, imperative package managers and associated distros in favor of the functional package management paradigm exemplified by Guix, but I still recommend openSUSE to my friends who prefer a more traditional/mainstream distro because of the love I have for the Open Build Service and Zypper.
The web interface for OBS does feel clunky these days, but it's a wonderful tool not just for improving the reliability and quality of software packages, but distributing them. Zypper is hands-down the most powerful and complete high-level package management tool I've ever used as part of a binary-based GNU+Linux distro. I love that openSUSE provides an instance of OBS that anyone can use for free to build packages for not just openSUSE but a TON of different distros.
I wish more people would explore, take advantage of, and celebrate OBS just like I wish they'd do the same with Nix and Guix!
I'm completely onboard with the idea of reproducible builds.
> The thing is docker is huge and bloated; is far from secure, and will probably stay that way for the foreseeable future
It would be a mistake to fully associate container workflows with docker itself.
https://github.com/genuinetools/img + https://github.com/opencontainers/runc can be used to provide almost the same workflow as docker, without using docker itself.
You could also use the '-f docker' option the post talks about with runc to run the resulting image in an unprivileged container.
It would also be a mistake to fully associate Docker containers with Docker.
Joyent can run docker containers, securely (zone-isolated), on smartos.
I use 'guix environment <somepackage>' and 'guix system vm config.scm' every day. I don't need more, cause these two solves most of the problems that was described earlier.
What is it I can provide that would be clearer, more common usage, than the examples I use almost literally as they are here?
And that 9 line docker file references at least one other unknown file, and is part of a bigger program. Docker would not be able to reproduce with the information given in that post. How do you expect me to reproduce something with at least 2 huge unknowns?
That is why you got a more generic answer for implementation, but once you have your implementation once, you only need the commandlines I provided.
So, the question is, how to package something with guix, and how to run it.
With docker you run something as docker run [--interactive] [--terminal] [--entrypoint=...] <image> [[command] args]
Your libsodium fork example is nice, but we still don't know how to package a simple program.
While it's neat that I can do introspection on my package graph, I don't immediately see any benefit for me when I startup my containers.
I would love to see a full guix/nix script of what GP asked to see a comparison, I like to see hands-on stuff not theoretical.
No, we show that Guix is a tool that gives you a way to work with software environments at a higher level; but at the same time you don't have to give up on application bundles like Docker. You can simply generate Docker images or other forms of applications bundles from that higher-level representation.
You are welcome to take a look at this paper that I co-authored where we explain why we use Guix for a reproducible bioinformatics pipeline, and the rigorous, declarative functional package management approach instead of the imperative approach of Docker files:
https://www.biorxiv.org/content/early/2018/04/21/298653
We're also providing Docker images, but we generate them from a higher-level declarative specification that ensures a high degree of bit-reproducibility.You define a package for your own project that depends on libsodium/zeromq/etc from GuixSD. Then you export your own package with 'guix pack'. For an example of what a package definition looks like, take a look in /gnu/packages in the GuixSD repository, for instance libsodium [1] or Vim [2].
I did something similar recently to build an Nginx "application bundle" [3]. It uses Nix (previously Guix, but Nix worked better for me in the end) to build a squashfs image. You can then run the binary on that filesystem with systemd-nspawn, or as a regular service by setting RootImage=. Some advantages over the Docker approach are that you can easily customise the build (e.g. changing the ./configure flags for Nginx without having to manually perform all other build steps), and bit by bit reproducibility (if you build the same commit six months from now, on a different machine, you will still get the same image out).
[1]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages... [2]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages... [3]: https://github.com/ruuda/miniserver#readme
FROM nixos/nix
RUN nix-channel --update
RUN nix-env -i python2.7-{twisted,treq,txgithub}
WORKDIR /app
ADD . /app
EXPOSE 8080/tcp
CMD python notify.py
The next level would be using the nixpkgs Docker builder directly: https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerToolsAnyway, in your Dockerfile I see that your application uses Python and you do some package management and service management stuff that is mixed together. In Guix, these things are separated. So the first step would be to define a package for your software, and then you would deploy that package. For a real world example of a Python application, here is what the AWS CLI package looks like:
(define-public awscli
(package
(name "awscli")
(version "1.14.41")
(source
(origin
(method url-fetch)
(uri (pypi-uri name version))
(sha256
(base32
"0sispclx263lybbk19zp1n9yhg8xxx4jddypzgi24vpjaqnsbwlc"))))
(build-system python-build-system)
(propagated-inputs
`(("python-colorama" ,python-colorama)
("python-botocore" ,python-botocore)
("python-s3transfer" ,python-s3transfer)
("python-docutils" ,python-docutils)
("python-pyyaml" ,python-pyyaml)
("python-rsa" ,python-rsa)))
(arguments
'(#:tests? #f))
(home-page "https://aws.amazon.com/cli/")
(synopsis "Command line client for AWS")
(description "AWS CLI provides a unified command line interface to the
Amazon Web Services (AWS) API.")
(license license:asl2.0)))
The package recipe contains all the metadata, build instructions, and dependencies. Now that you have a package, it can be built with Guix and then deployed in a variety of ways. Judging from the Dockerfile, your software is some daemon that listens on port 8080, so:* You can install the software directly using 'guix package -i your-package-name' and run the notify.py program. Good for trying things out.
* If you are deploying to the Guix system distribution, you could write a service definition so that you can manage the daemon via the init system. The service would take care of creating the notifier user and group, starting the service on boot, etc.
* You could use 'guix pack --format=docker' to export an image suitable for running with 'docker load'
* You could use a different 'guix pack' format (and maybe make it relocatable) for running on some other non-Guix system
I should also add that I don't think the work is fully done yet on handling the entirety of Docker use-cases. It's a work in progress. I can think of a number of things that I want to add to Guix to make this workflow better that I haven't had a chance to hack on yet.
If the versions are specified in the 'python-botocore' type definitions, how do you install more than one version of a library?
Does guix only track the latest version of dependencies or can you request any version of something?
The package here uses the `python-build-system`, which defaults to the latest version of Python, but you can override that by specifying `(arguments '(#:python ,my-python))`, where `my-python` is a variable bound to a package value of the Python variant that you want to use.
You can easily install more than one version of a package as long as you have a package definition for it. You can install different variants (not just different versions) into separate profiles.
Guix is a Scheme library providing lots of variables that are bound to package values. These package values may have links to other packages (that's done with quasiquotation). Together they form a big graph of packages with zero degrees of freedom. Every version of Guix provides a slightly different variant of this package graph. When installing any package you instantiate a subset of this particular graph. Updating or modifying Guix gives you a different graph.
In order to keep things manageable we try to keep the number of variants of any particular package in Guix to a minimum, but you can install older variants by using an older version of Guix; or you can add new variables that are bound to package variants or different versions and install those.
It's very convenient and conceptually simple.
Then it would be great to see t0nt0n or someone else who knows guix do the port so we can fully compare these two approaches.
requests==2.18.4
the actual packages generally aren't important.The cases were that would become interesting are where they require some C library dependencies first, like libpq-dev. In those cases something like guix/nix would be nice because it could be used to pull in the specific external dependencies as well.
Do you want to convince people that something like guix is better than docker? Then take something that is currently distributed using docker and actually show how the guix approach is simpler.
i.e. I have a random app I recently worked on where the dockerfile was something like
How do I actually take a random application like that and build a guix package of it?Another project I work on is built on top of zeromq, and it would be great to use something like guix to define all the libsodium+zeromq+czmq+zyre dependancies and be able to spit out an 'ultimate container image' of all of that, but all this post shows me how to do is install an existing guile package.