There is no finer system for turning quick CI hacks into entrenched technical debt.
Jenkins gives you the freedom to do things without the burden of asking whether you should do those things. Jenkins permits you to munge a whole lot of separate domains into a single, creaky system.
There are better tools available for most aspects of CICD and automated testing these days.
It's easy to say this and not cite examples.
I rock perforce and windows agents, requirements of the environment, and we struggled very hard to find something that beats Jenkins.
One guy even bought an Octopus Deploy license for a year (at the tune of a few thousand dollary-doos) and it did everything worse than our Jenkins setup.
I actually dislike Jenkins, I'm open to suggestions here.
Since we just had set Gitlab up as our source control platform, this was the logical choice and we immediately ripped down the Jenkins server. Since we were still starting out, integration and rewiring was done in just a day.
I‘ve never been so happy playing the early adopter guinea pig. Actually we never had any real issues with it and aren‘t looking back.
For me, powerful equals "footgun included". CI should do one job and do it well imho. Huge bonus points for tight coupling into the source control workflow.
Your mileage may vary.
Things I've found better than Jenkins once an organization lets go of over-engineered processes:
- TeamCity - Concourse CI - Circle CI - ArgoCD - Tekton
For Windows build agents I've met plenty of people successful with Octopus Deploy that have hated Jenkins and your data point is the first I've seen that went back to Jenkins.
Perhaps it's worth looking at Team Foundation Server / Azure DevOps?
The alternatives listed in reply to your thread are all much better than Jenkins, from system design, to usability, to cost of ownership of the personnel needed and tech debt incurred. Here's my own opinionated list, having used or evaluated them before:
1. GitlabCI 2. Circle CI 3. Travis CI 4. Github Actions 5. TeamCity 6. Concourse CI
Jenkins just happen to have been around a long time. It's like Jira -- everyone uses it, but nobody likes it.
I think it still has no rival in that area and it's also why is getting a bad rep.
They don't get access to your code (BYO runners, they manage pipeline config / logs / notifications). You run their (open source, easy to build yourself) agent on infrastructure you control.
It's really nicely put together.
Jenkins tends to suffer from extremely poor state management and experimenting with things can cause a permanent loss of value to companies if backups are not properly configured - compared with a system like ansible where recipes will grow and be regularly committed to a repository Jenkins doesn't have a nature version control system and as a result strikes me as extremely brittle.
I almost wish the UI for jenkins simply didn't exist and it actually was just a whole mess of config files and shell scripts so that it could be locked down to a much better degree.
However, when coupled with Kubernetes, Jenkins is more powerful than almost any other CI tool.
You can orchestrate any iteration of build types or actions while easily providing the underlying resources transparently to your users.
Notice that almost every other CI tool mentioned here is either 3rd party hosted or severely limited in some fashion when compared to Jenkins.
Gitlab Action Services for instance is closer to Docker Compose.
Regarding your batch scheduling solution - take hashicorp nomad for a spin: a single go binary and your scheduled job can be declared in 10 lines of yaml. Won’t miss a beat.
Stop ~~~ engineering ~~~ things and choose the right tool for the job.
A large portion of the cost of many of these tools is spent "choosing the right tool for the job" Figuring out what they do, what they do well, where they overlap, where the company that makes the tool is headed, how hard its going to be to swap that tool out for a better one (or cheaper one) - thats a lot of expense in labor and training.
I claim using pretty much any tool other than jenkins is easier to maintain over time (as illustrated in this very article).
An ability to share machine pool with a CI system, or having an integrated CI is a plus.
We don't use Jenkins for CI/CD. It's probably too overkill for those basic tasks.
But as the middle ground between crontabs and something like Airflow it fixed all our problems without creating new ones.
Are there really any alternatives out there? (And no, we don't need Kubernetes, thanks.)
Most folks I've seen replacing Jenkins for a job runner wind up using Ansible, Salt, or Rundeck if they're not in a giant enterprise (there's also the old HP Operations Orchestration system but that's buried now unfortunately). All of these have their own holy wars brewing and deficiencies, sure, but I personally prefer those warts over adding more maintenance issues by adding in Jenkins. I'd also suggest StackStorm for a more modern, async approach to orchestration workflows. The workflow software ecosystem is built around business domain specialization and it's kind of silly to try to go against Jenkins toe to toe as a company, so this is what we've got sadly. After all, Jetbrains makes TeamCity which is essentially a Better Architected than Jenkins system but it's definitely not very popular either.
- Website with access control which supports identity provider.
- Properly authorized user can start a job from their browser, and can specify arguments as well (using checkboxes, input boxes, multi-choice boxes etc...).
- Job works by allocating few machines from the pool and running some shell commands on them. The number of machines, type of machines, and commands to run are all customizable and can vary based on parameters user has entered.
- Once job is running, you can view text logs (in real time) or hit "cancel" button, and it'll properly cancel job and release all the resources. If the job fails for any reason, it sends notifications.
- From web interface, you can view the list of all past jobs, their status, logs, outputs, test results, etc...
- The machines can be physical on-prem ones, or allocated from AWS. For physical machines, all you need is plain linux with ssh access. For AWS machines, machines are created/destroyed in response to load.
- The whole thing needs no infra other than a single master machine which is hosted on premises. As long as you back up the master regularly, you can recover from complete meltdown, the only problem will be that some jobs will get cancelled.
------
Jenkins is actually pretty bad at this task, and we need a fair amount of Groovy code (ugh...) to get it to do what we want.
But there seems to be very little alternatives which can do all that. For example, Apache Airflow is missing authentication for web ui. Tekton seems to require Kubernetes and I (based on reading docs) has no UI controls. etc...
Jenkins is architecturally an artifact of enterprise software kitchen sinks for the 2000s and the way around it is to cut away at anything one doesn’t need out of Jenkins and to instill discipline in engineers to think about requirements carefully and to avoid hacking more and more into Groovy Jenkins libs than one would spend time actually doing work.
For a lot of what you’re describing Rundeck was built for this common case as essentially a central job portal, and at the least it supports more integration options than Jenkins. It’s difficult to wholeheartedly recommend it still given its surprisingly low rate of adoption. The market now is that for an OSS product a decent web UI with authentication is basically something to charge like it’s an enterprise product, and so lots of start-ups even today will begrudgingly fire up Jenkins in 2021.
I’m of the opinion that the time fighting Jenkins jobs and writing Groovy for common, trivial tasks in most CI systems is not acceptable in a start-up situation where strong focus and minimizing distractions / non-core work is so important for success. The kind of code to define a build doesn’t make sense for orchestration (it’s not just a digraph but about how to react to events and edge transitions) but it’s exactly how Jenkins DSLs work out.
There's no comparison between managed/logged/permissioned/distributed jobs that jenkins provides for free vs building an overwrought service or an insufficient crontab. However it's a power-tool and I think a lot of people go in expecting something dead-simple and pretty and are put off by something that they need to invest in learning.
Just for example, in 2 days I built an uptime checker that ran every 60-seconds via jenkins and triggered slack alarms/sms on problem, and it just worked (< 2 days total maintenance), for years. An equivalent service (Pingdom) quoted 10k a year.