Second this. I really would rather have something I can configure fully with YAML or JSON.

Vespene is pretty new so a lot of things are up for grabs. Pipelines were implemented in about a day, so the potential to make rapid upgrades is pretty big.

That all being said, I think you have 95% of that right there today with http://docs.vespene.io/importing.html

This is a JSON file, and it can define what pipeline a project is part of, and what stage it is in that pipeline.

So it's like "this project is part of the analytics pipeline, and it is part of the deployment stage".

But you still have to make the analytics pipeline in the GUI right now. This means that you have to say (and this is all) the analytics pipeline has the following stages in order: A, B, C, and then D.

I think it's quite possible to have the definition OF the analytics pipeline in the vespene.json too, but it's a bit of a question of which one wins if there are conflicting definitions.

If you want to stop by talk.vespene.io this would be a great thing to bounce ideas around on!

I think https://concourse-ci.org/ is something you want to look at then.

Concourse has a really weird resources philosophy that makes it almost impossible to easily work with.

I'm a notorious Concourse fan, I'd be interested to understand why you think it's almost impossible to work with.

Weird in what way?

VSTS/Azure Devops is pretty good for this, as long as you host your own build servers. Their hosted solution is terrible.

All can be built through their UI, then exported to yml in your repos. Pretty good UI. Their service connections are good.

Jenkins has plugins so you can version control the job configs in your RCS. Jenkins pipeline or job-dsl2 (older)

Yes and those are great when the people managing the Jenkins server are willing to install those plugins and get everyone to rewrite their stuff to use that. They did that eventually install that at $WORK, but they have a bunch of configuration for it for various libraries, which they make backwards-incompatible changes to without warning.

Fortunately, we're mostly moving to GitLab CI which doesn't have most of this bullshit.

It’s worth noting that GitLab CI will turn your Yaml build config into a nice graphical pipeline

(Another fan here)

Jenkins’ Blue Ocean also displays pipelines constructed in Groovy.

We are glad to hear that you like using GitLab CI. What are your favorite features?

Good documentation, build artifacts, good integration with rest of GitLab.

we use https://wiki.jenkins.io/display/JENKINS/JobConfigHistory+Plu... to track job config changes. Additionally we require users to checkin the config.xml for each job in VCS- It's easily accessible by appending config.xml to the Job URL eg. http://server/job-name/config.xml

That seems to take care of who changed, what changed, etc.

At an earlier job we installed a VC history plugin (I don't know which one) to at least have basic history of configuration changes, but it had several problems:

- It would often get into an unrecoverable state, and we would have to SSH into the box to clean things up to allow it to continue (and in the meantime we would lose all history).

- Trivial automated changes from another plugin would flood the log. Not this plugin's fault, of course, but Jenkins is unusable without lots of plugins, most developed in isolation from others.

Everyone I've talked to about this prefers the GitLab model of telling the server everything relevant to your build at build time. Where Jenkins would need copies of jobs or synchronized configuration updates, GitLab just assumes that each commit has the job configuration it needs.