Bazel/Gradle/Maven/etc tend push you in the right direction, bash, etc dont.
Or to put it succinctly, you need both a build system and a CI system, they are not the same thing.
> Fix CI issue with blah blah blah
> Hmm that didn't work lets try something from stackoverflow
> Build fix
> Build fix please work
> Please
> I hate my life
> I am tired and hungry, I want to go home
> Stupid yaml
The CTO would call him like a week later to make sure he was okay. This happened often before we switched to Nuke. It rarely happened afterwards. It's awesome being able to debug your CI locally.
Do you wan to migrate all your third party libs to Bazel, or just hack up a few lines of CI yaml to call the authors build system?
I lean on gitlab yaml quite heavily and it feels quite effortless
Personally I use the native build system that upstream uses for most of them, and just hack in my compiler flags/options where necessary. Then I use CI triggers to rebuild my projects when the dependencies are rebuilt.
Yeah you need a k8s cluster, but even a simple kind dev cluster that you spin up in 30 seconds with one command on your laptop will work.
I like it a lot because it enforces very little structure on you and doesn't reinvent everything. Stuff like storage (either ephermeral or existing volumes), secrets, configuration, etc. are already modeled and supported by Kubernetes and tekton can use all of that natively. And since it's all k8s native stuff you have all of the power of k8s, like its entire API for manipulating and managing execution, exposing services, etc. There is very little cognitive overhead or new things to learn once you know k8s.
If you're really averse to k8s though, check out drone. It has a local execution mode that is similar and just runs whatever pipeline commands you want in docker containers. https://github.com/drone/drone Batect is another even more minimal tool that's effectively just a docker-based workflow system: https://github.com/batect/batect
For instance, if running a bunch of parallel tasks, collating results on a PV is out the window unless your cluster supports multiple writer volume types, which GKE does not. You have to bring in NFS volume types or something like that for it. In the early days of tekton they had a results primitive which synchronized an output dir to GCS, but they decommissioned that. So you are left pushing that logic into your task command. Running gsutil is easy enough, but it means you are pushing logic into your scripts and not declaring steps in the pipeline definition. You could make that command a step but I see little benefit in that.
Additionally there is no way to loop in the configuration to generate tasks, much less loop with an ordinal value. We end up just programmatically generating the resource definitions with ruby erb templates. All of our pipeline specs (including task runs, etc) creates a 2MB yaml file. We push dozens and dozens of these into k8s daily. It works but at the same time our usage of Tekton is more or less as a glorified alternative to batch jobs which works because batch jobs _still_ don't have a proper sidecar capability and also because we rely on the DAG to order dependent taskruns.
If your pipeline is simple, look at Tekton. But if your pipeline is complex... still look at Tekton but expect to do some work. Once you get a good workflow though you can you can scale your pipelines as easily as you can a deployment in k8s. We use node autoscaling and preemptibles (Tekton can retry if a task disappears due to node reclamation) to manage our CI costs quite effectively.
Some other comments here have argued for pushing as much logic as possible into your scrips, so that they can be tested without the CI system. What's the downside of doing this?
Every tool I've used allows you to do this. Just write the script in the language of your choice, then run it in the pipeline.
A pipeline should be considered like "markup" around your scripts. Everything should work independently and on any system. The pipeline config just tells the scheduler which order to run it in, which bits can run in parallel, what artifacts to keep etc.
If gitlab/GitHub could just provide hooks to do this stuff for a range of scripting languages that could be easily dry run this would solve the problem.
My solution is: https://github.com/rosineygp/mkdkr
I can write somethings like this:
py:
@$(dkr)
instance: python:3.8
run: pip install requests
export MKDKR_SHELL=python
run: 'import requests
r = requests.get(f"https://api.isevenapi.xyz/api/iseven/2/")
print(r.json())'Want to do something fancy and complex? Just pull in a python image and call your python script. But you can still implement simple build steps directly in the build file. No need for a multitude of language bindings, everything you want to do is probably implemented with simple shell commands.
Nuke [1] gets close but there are still a lot of tasks that don't have C# bindings, such as publishing build artifacts and uploading test results.
While I'm dreaming about my perfect CI, I'd also like the ability to download benchmark results from previous commits so that I can generate trend graphs and publish them in the build results.
To do this right the CI system would have to have an API using REST, GraphQL, gRPC or some such API format that generates clients in many languages. That way they don't have to maintain bindings in every language.
[1] https://nuke.build/