marcosnils

can confirmed it's fixed in nightly!

> but I'm not convinced it's ready to replace Gitlab CI.

The purpose of Dagger it's not to replace your entire CI (Gitlab in your case). As you can see from our website (https://dagger.io/engine), it works and integrates with all the current CI providers. Where Dagger really shines is to help you and your teams move all the artisanal scripts encoded in YAML into actual code and run them in containers through a fluent SDK which can be written in your language of choice. This unlocks a lot of benefits which are detailed in our docs (https://docs.dagger.io/).

> Dagger has one very big downside IMO: It does not have native integration with Gitlab, so you end up having to use Docker-in-Docker and just running dagger as a job in your pipeline.

Dagger doesn't depend on Docker. We're just conveniently using Docker (and other container runtimes) as it's generally available pretty much everywhere by default as a way to bootstrap the Dagger Engine. You can read more about the Dagger architecture here: https://github.com/dagger/dagger/blob/main/core/docs/d7yxc-o...

As you can see from our docs (https://docs.dagger.io/759201/gitlab-google-cloud/#step-5-cr...), we're leveraging the *default* Gitlab CI `docker` service to bootstrap the engine. There's no `docker-in-docker` happening there.

> It clumps all your previously separated steps into a single step in the Gitlab pipeline.

It's not generally how we recommend to start, we should definitely improve our docs to reflect that. You can organize your dagger pipelines in multiple functions and call them in separate Gitlab jobs as you're currently doing. For example, you can do the following:

  ```.gitlab-ci.yml
  build:    
    script:
      # no funky untestable shellscripts here, but calls to \*real\* code
      - dagger run go run ci/main.go build
  test:
    script:
      # no funky untestable shellscripts here, but calls to \*real\* code
      - dagger run go run ci/main.go test
  ```

This way, if your pipeline currently has a `build` and `test` job, you still keep using the same structure.

> but is very annoying if you use Gitlab CI's built in parsing of junit/coverage/... files, since you now have extra layers of context to dig trough when tests fail etc

You can also still keep using these. The only thing you need to be aware is to export the required test / coverage output files from your Dagger pipelines so Gitlab can use them to do what it needs.

> but I've just written quick-and-dirty scripts to do that every time I've needed it.

This is what we're trying to improve. Those quick-and-dirty generally start very simple but they become brittle and very difficult to test by other engineers. Yes, you could use docker or any container-like thing to enable portability, but you'll probably have to write more scripts to glue all that together.

Quoting one of our founders:

"Our mission is to help your teams to keep the CI configuration as light and "dumb" as possible, by moving as much logic as possible into portable scripts. This minimizes "push and pray", where any pipeline change requires committing, pushing, and waiting for the proprietary CI black box to give you a green or red light. Ideally those scripts use containers for maximum reproduceability. Our goal at Dagger is to help democratize this way of creating pipelines, and making it a standard, so that an actual software ecosystem can appear, where devops engineers can actually reuse each other's code, the same way application developers can."

Hi there, Dagger contributor here. We're solving this exact same problem by allowing you to encode your CI/CD pipelines in your favorite programming language and run them the same way locally and/or any CI provider (Gitlab, Github Actions, Jenkins, etc).

We're very active in our Discord server if you have any questions! https://discord.gg/invite/dagger-io

> I completely agree, which is why Dagger is perfect for medium/large projects.

Initially that was the case, but with the upcoming work on modules (aka project Zenith), hopefully this changes. After all, IMO the ultimate aspects that will make it super easy to adopt is the ecosystem and learning how to use the tool. The later is mostly tackled by the "pick your favorite language" story; the former is where we hope modules can help users the most.

Dagger fits very nice with small and large dev and devops teams. Similarly to other tools, the more you grow, the better you'll have to organize your projects to make the best use of it.

Hi there, Dagger contributor here.

One thing that generally is not immediately obvious is all the side benefits that using a programming language actually unlocks.

Need type checking at build time? done.

Need a package manager to version and publish your CI code? done.

Need linting / formatting? done.

Need to import functions from other places? done

Need a test framework? done.

Need a better collaborative IDE? done.

All these things above bring massive improvements to both Dev and Devops engineers that need to interact with build & deploy pipelines daily. Generally it'll be quite difficult for any other configuration or custom language to provide these kind of tooling.

Hey there, Dagger employee here.

We've heard this feedback from a bunch of our users, and even though the awesome work of nektos/act has helped some of them, we feel like there might be more to just try to emulate GHA locally. After all, GitHub's doesn't have any incentives to allow their users escape their platform.

Having said that, our community has pushed the boundaries of Dagger and has made awesome progress on streamlining GHA into our platform. Here's two in-progress demos about some of that work

Source to action: https://youtu.be/EgOvSbwKVts Running GHA in Dagger (project gale): https://youtu.be/rHk3spfCHFQ

We could use some feedback and help to make this even better. If you feel like that might be interesting, we're very active in our discord server: https://discord.com/invite/dagger-io

Hey, Dagger employee here. This exactly is one of the the main design decisions of Dagger. We're not expecting companies and/or projects to perform a full migration to it, that's why we generally recommend starting by wrapping different parts of your pipeline and move on from there. Unlike other solutions, Dagger allows you to keep using your existing tooling and reuse Dockerfiles (https://docs.dagger.io/quickstart/429462/build-dockerfile/#r...) while slowly transition your pipelines to code.

TIL about Jeeves, things like something that could play very nice with the Dagger python SDK (https://docs.dagger.io/sdk/python). Particularly since Dagger could help with bootstraping the python runtime and providing caching capabilities out of the box.

Just to clarify, Dagger is both declarative and imperative. Yes, you can use multiple programming languages which exposes an imperative API, but your pipeline DAG has to be defined using Dagger's SDKs which is has a declarative form.

Hey there! Dagger employee here. Congrats on the launch, it's grateful to see other people trying to collaborate on this space.

> Such as dependency management (in the case of python, the requirement of a python interpreter, and its package managers, etc). togomak is just a single statically-linked binary.

Not sure if you've been following the latest updates about Dagger, but since the latest 0.9 release we have a preview feature where all the SDK tooling runs within Dagger itself as a container so they're not required to be installed in the host machine anymore. We've also added support for Dagger Modules which allows you to package and distribute pipelines a-la terraform. Our docs are still not updated yet but if you join our Discord server, you can find all the information there.

Cheers!

Hey, https://dagger.io employee here. We can help with this. I'd recommend taking a look at out Python SDK (https://docs.dagger.io/sdk/python) to build your pipelines and then leverage Dagger Cloud (https://dagger.io/blog/dagger-cloud) for fast production deployments. Happy to help if you have any questions.

Thx for reporting. We'll fix ASAP

Github actions with https://dagger.io is simply awesome!

Sounds a lot like what systeminit is working on: https://www.systeminit.com/

Have you seen https://dagger.io/? Managing all that with code has been a life changer to me.

Great article! thx for sharing. I'm currently contributing to Dagger(https://dagger.io) and this looks like a great idea to generalize as a Dagger library. By any chance did you think about it?

:wave: Dagger contributor here. This is exactly one of the many reasons why we've decided to build Dagger. We're working for a future where CI pipelines can be defined in your language of choice and composable across different ecosystems. If you want to know more, happy to chat in our Discord community.

https://dagger.io/