News for hackers. Stuff that matters.
Re:Ko: Easy Go Containers by hknmtt on Wednesday November 08, 2023 @02:38AM (#38187694)
I have used something like this in the past:

FROM alpine:latest AS base # Scratch does not have shell so we have to create non-root user in here and later copy it into scratch. RUN adduser -D -u 123456 appusr

FROM scratch # Copy the binary. COPY foo.bin /foo.bin # Copy the user and set is as active. COPY --from=base /etc/passwd /etc/passwd USER appusr # Non-root user cannot bind to "privileged" ports(<=1024). EXPOSE 1234 ENTRYPOINT ["/foo.bin"]

Simple. But i can see ko being good alternative if you for some reason do not want to install docker on your computer but still be able to build docker containers.

Re:Ko: Easy Go Containers by traceroute66 on Wednesday November 08, 2023 @02:51AM (#38187779)
> i can see ko being good alternative if you for some reason do not want to install docker on your computer

The already established good alternative to Docker desktop is Rancher Desktop[1].

[1] https://rancherdesktop.io/

Re:Ko: Easy Go Containers by jpgvm on Wednesday November 08, 2023 @02:56AM (#38187815)
This is not talking so much about the Docker GUI things but rather this doesn't require a Docker daemon at all.

Similar tools are Google's Jib for JVM based applications, `rules_docker` and `rules_oci` for Bazel etc.

All of which allow you to build applications without needing a Docker runtime at all during the build process which is a huge plus for portability, performance and repeatability.

Re:Ko: Easy Go Containers by gtirloni on Wednesday November 08, 2023 @05:52AM (#38188754)
Re:Ko: Easy Go Containers by ricardbejarano on Wednesday November 08, 2023 @06:29AM (#38188979)
Re:Ko: Easy Go Containers by BossingAround on Wednesday November 08, 2023 @07:03AM (#38189264)
From my experience, podman desktop is pretty good at containers, but horrible for Kubernetes (it uses KIND but you can't configure basic things, like exposing NodePort IIRC).

Given that SUSE uses Podman, I hope they'll modify Rancher Desktop to use Podman, in which case it'd supersede Podman Desktop.

Re:Ko: Easy Go Containers by BossingAround on Wednesday November 08, 2023 @07:04AM (#38189284)
Sadly, Rancher Desktop still uses Docker, right?
Re:Ko: Easy Go Containers by jadbox on Wednesday November 08, 2023 @06:51AM (#38189157)
I thought that go doesn't work in Alpine because of shared libs?
Re:Ko: Easy Go Containers by jstanley on Wednesday November 08, 2023 @07:57AM (#38189766)
Build with `CGO_ENABLED=0`, perhaps?
Re:Ko: Easy Go Containers by ImJasonH on Wednesday November 08, 2023 @08:37AM (#38190095)
That works, but it also means maintaining a Dockerfile that does the COPYs, when all you're really doing is assembling image layers. This is more or less exactly what ko does, without using the Dockerfile language to describe it.

Not having to run a Docker daemon is just a nice bonus! :)

Re:Ko: Easy Go Containers by 015a on Wednesday November 08, 2023 @09:25AM (#38190687)
Or, if you don't want to install it in your CI/CD. Which could be extremely valuable; some CI/CD providers make "building a docker image" a weirdly hard (and sometimes, even much more expensive) step due to docker-in-docker madness.
Re:Ko: Easy Go Containers by matrss on Wednesday November 08, 2023 @10:27AM (#38191755)
In my experience, not using docker to build docker images is a good idea. E.g. buildah[0] with chroot isolation can build images in a GitLab pipeline, where docker would fail. It can still use the same Dockerfile though.

If you want to get rid of your Dockerfiles anyway, nix can also build docker images[1] with all the added benefits of nix (reproducibility, efficient building and caching, automatic layering, etc.).

[0] https://buildah.io/ [1] https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-dockerToo...

Re:Ko: Easy Go Containers by chickenbane on Wednesday November 08, 2023 @02:25PM (#38195608)
> But i can see ko being good alternative if you for some reason do not want to install docker on your computer but still be able to build docker containers.

Docker is a much bigger dependency than ko, involving a daemon and a socket you have to manage and secure. ko is a only a go program, and builds are straight-forward and lightning fast compared to Docker.

Importantly, ko also removes the need for a Dockerfile. Yes, as you point out a Dockerfile can be simple, but but the best part is no part. Dockerfile has plenty of foot guns and best practices to learn so if at the end of the day all you need is a go binary on a container ko will serve you much, much better.

Re:Ko: Easy Go Containers by ithkuil on Wednesday November 08, 2023 @05:12AM (#38188530)
I love `ko`. I wish there was something similar for rust but I haven't found it quite yet
Re:Ko: Easy Go Containers by bambambazooka on Wednesday November 08, 2023 @07:26AM (#38189491)
I have a very opinionated library that builds Docker images in pure Rust.

I don't think building something like `ko` with it should be that much work. (I know, famous last words)

I'll have to check with $employer if it's ok to open source it.

Re:Ko: Easy Go Containers by jokethrowaway on Wednesday November 08, 2023 @11:17AM (#38192562)
That sounds amazing, please do!

Ko sound like a great idea but I have little interest in running go services!

Re:Ko: Easy Go Containers by jppittma on Wednesday November 08, 2023 @08:44AM (#38190185)
Re:Ko: Easy Go Containers by ImJasonH on Wednesday November 08, 2023 @11:05AM (#38192362)
rules_docker is deprecated in favor of rules_oci, which also has Rust support: https://github.com/bazel-contrib/rules_oci/blob/main/docs/ru...

I think Bazel can be a good fit for larger polyglot organizations that need to manage large codebases in many languages in a uniform way. Basically Google-circa-2010-sized organizations, coincidentally!

For smaller teams, adopting Bazel too early can be a real productivity drain, where you get all of the downsides of its constraints without as many of its benefits. Bazel is overkill for a project of ~10 Go apps, for example. Ko was actually created to help such a project (Knative) migrate off of Bazel's rules_docker to something better, and I think it achieved the goal!

Re:Ko: Easy Go Containers by ImJasonH on Wednesday November 08, 2023 @08:44AM (#38190196)
I'd love to build a ko-for-Rust. I'd just need to learn Rust a bit more. :)
Re:Ko: Easy Go Containers by evntdrvn on Wednesday November 08, 2023 @09:57AM (#38191237)
There's lots of people that would be willing to help!!
Re:Ko: Easy Go Containers by matrss on Wednesday November 08, 2023 @10:47AM (#38192051)
If you have your project packaged with nix (which shouldn't be too complicated if your build process is not too special) it is trivial to turn a nix derivation into a docker image: https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-dockerToo.... As a bonus, it is completely language-agnostic.
Re:Ko: Easy Go Containers by riv991 on Wednesday November 08, 2023 @05:17AM (#38188564)
I love ko, so simple to get a container running in one line and now integrated with GoReleaser.

`docker run $(ko build -L main.go)`

Re:Ko: Easy Go Containers by Cyphase on Wednesday November 08, 2023 @07:24AM (#38189459)
For those who don't know: https://en.wikipedia.org/wiki/Ko_fight
Re:Ko: Easy Go Containers by ImJasonH on Wednesday November 08, 2023 @08:43AM (#38190169)
Hey, ko maintainer here! I'd love to answer any questions or hear any feedback folks have.

Ko's simplicity comes from focusing on doing exactly one thing well -- it doesn't have to run containers to build code in any language, it just builds minimal Go containers, and that means it can focus on doing that as well as possible.

Another powerful benefit of focusing on Go (IMO) is that ko can be used to transform a Go importpath to a built image reference.

A common simple use: `docker run $(ko build ./cmd/app)`

This is also how Kubernetes YAML templating[1] works, and it's also how the Terraform provider[2] works -- you give it an importpath, and ko transforms it into a built image reference you can pass to the rest of your config. I don't even "use ko" day-to-day most days, I mostly just use terraform. :)

1: https://ko.build/features/k8s/ 2: https://ko.build/advanced/terraform/

Re:Ko: Easy Go Containers by tuananh on Wednesday November 08, 2023 @08:53AM (#38190288)
Hello Jason!
Re:Ko: Easy Go Containers by anuraaga on Friday November 10, 2023 @04:12AM (#38216743)
Ko works ok but lacks a lot of configurability, compared to something like jib, that does end up important. There are a few open issues on it but the commit history is basically dependabot, not much action for UX. Better than nothing I guess but it really does give off the feeling of a project spun off from Google to let it die.

Despite that it's probably still the best tool for building containers in Go, but it's not pleasant.

Re:Ko: Easy Go Containers by pachico on Wednesday November 08, 2023 @08:57AM (#38190336)
Dammit, why didn't I know about this? In general, I always have the feeling I'm not aware of great applications out there, and I confirm this sensation every time I find them, like now!

How do you all stay tune of the great apps out there?

Re:Ko: Easy Go Containers by seanw444 on Wednesday November 08, 2023 @09:40AM (#38190930)
I track GitHub's trending page for Go, Rust, and occasionally Python projects. Limiting to these languages tends to yield the kinds of projects I'm interested in, just by the nature of the demographic that uses them. Sometimes I browse recent repo activity on Sourcehut too.
Re:Ko: Easy Go Containers by Alifatisk on Wednesday November 08, 2023 @09:10AM (#38190491)
Wondering the same
Re:Ko: Easy Go Containers by avtolik on Wednesday November 08, 2023 @09:36AM (#38190871)
I am building an app right now and looked into this. Unfortunately the support for including static assets is limited for my case.
Re:Ko: Easy Go Containers by ImJasonH on Wednesday November 08, 2023 @09:53AM (#38191175)
What's your use case? Static assets are fairly limited today, but even within those limitations we've been able to get pretty far with symlinks.

If that's not enough or not desirable I completely understand, there's probably something we could do to improve it.

https://ko.build/features/static-assets/

Re:Ko: Easy Go Containers by arccy on Wednesday November 08, 2023 @11:08AM (#38192408)
depending on what sort of support you need, you can also consider embedding the static assets into the go binary
Re:Ko: Easy Go Containers by flimsypremise on Wednesday November 08, 2023 @10:19AM (#38191593)
I'll be honest, I just don't think this is a great way to do development in any language:

`ko builds images by executing go build on your local machine`

If you've done any sort of work with service or application development on a team, you will no doubt have encountered issues with relying on local machine build dependencies. Enforcing dependency versions across the team is basically impossible even in cases where the entire team is using the same OS and hardware, so you're going to run into problems where users are building the codebase with different versions of tools and dependencies. This is one of the core problems that containerization was intended to solve, so if you are using containerization in development and not building inside the container, you are missing out on one of the major advantages of the paradigm.

Re:Ko: Easy Go Containers by ImJasonH on Wednesday November 08, 2023 @10:41AM (#38191950)
I do agree with you in general. However, while containers were being invented to solve this problem, Go was also solving this problem.

For the most part, for the most common simple Go applications, if you build the same code with the same version of Go installed, you'll get the same dependencies and the same artifact.

Building Go applications in containers is not necessary in general, and doing so makes it much more complicated to share a build cache. You can of course do it with enough additions to your Dockerfile, but why bother?

If your developer team and CI environment are all using a similar recent Go, they shouldn't have different behavior using ko together.

Re:Ko: Easy Go Containers by flimsypremise on Wednesday November 08, 2023 @11:35AM (#38192834)
See now that right there is the issue. _IF_ your developer team and CI environment are all using a similar recent Go. Maybe you can keep this going with a single application and a small team that is actively maintaining that application, but even then I've run into issues with people using different computer with different architectures. Just this past week I had to solve a problem with a Go repo that wasn't building a container caused by the fact that whoever built out the development configuration was using an Intel Mac and certain aspects of the build didn't work on M1/2 Macs. I solved this by containerizing the entire thing, and now the entire setup for repo is a single command. No need to install any dependencies aside from Docker on your machine, no wondering if you're following the instructions correct, just clone the repo and `make run`.

Now extrapolate this out to larger orgs, where months might pass between changes to repos, and commits might be coming from disparate teams. Trying to mandate specific Go versions AND versions of any developer tooling that you might need to run your Go application locally, and you are inviting chaos.

Always build in the container. Local system dependencies are the enemy. Docker configuration for Go is actually extraordinarily simple compared to most languages, and even something like build caching is easy to handle via volumes.

Re:Ko: Easy Go Containers by matrss on Wednesday November 08, 2023 @12:32PM (#38193866)
I agree with this being an issue, although certainly less pronounced with go compared to other languages. I don't think containers solve the issue though: none of the mainstream approaches to building containers actually make the builds (and by extension images) inherently reproducible. It just shifts the issue from differing host systems to differing base layers (granted, the issue is less pronounced with those). As long as not every dependency is fully pinned your builds could break any day and arbitrarily between machines, if you don't build at least at the exact same time.

To solve the issue fully you need a more comprehensive approach to packaging. nix or guix can provide that.

Containers are more useful as a software distribution mechanism.

Re:Ko: Easy Go Containers by flimsypremise on Wednesday November 08, 2023 @03:34PM (#38196567)
I would say that containers are the foundation of the practice of reproducible builds. They don't solve the problem on their own, but containerization is a core element of the best practice of reproducible builds, along with utilization of lockfiles and infrastructure as configuration. It's certainly a hell of a lot easier to get a containerized application to a reproducible state than one run on a local machine with an unknown architecture and OS.
Re:Ko: Easy Go Containers by matrss on Wednesday November 08, 2023 @04:03PM (#38196978)
> I would say that containers are the foundation of the practice of reproducible builds.

If you mean containers as in the isolation features that are utilised by docker et al. to provide their flavor of compartmentalization then yes, those are pretty useful for reproducibility (although not 100% necessary, I think).

If you mean containers as in a bundled linux userland (so, e.g. a docker container running some image) then no, that is entirely orthogonal to reproducibility, as demonstrated by nix and guix which AFAIK use the low-level isolation features for their build sanboxes but do not have anything resembling a container image involved.

Re:Ko: Easy Go Containers by pjmlp on Thursday November 09, 2023 @05:49AM (#38203383)
Containers predate this common GNU/Linux problem and weren't invented to fix it, rather sandbox applications.
Re:Ko: Easy Go Containers by arp242 on Wednesday November 08, 2023 @10:46AM (#38192027)
Unlike languages that take a lot of stuff from /usr/lib/ system directories, this is not a huge problem in Go, as it doesn't do that. I'm not saying it's never a problem; but it's fairly rare