News for hackers. Stuff that matters.
Re:Faster and more flexible pipelines with a Directed Acycli... by qppo on Tuesday May 12, 2020 @11:11AM (#23154821)
I'm confused, a DAG is a model for the relationships between data. What makes this a separate feature for the product? And it's pretty fundamental to the problem you try to solve, usually, so what were they using before, a list? Tree?
Re:Faster and more flexible pipelines with a Directed Acycli... by chrisseaton on Tuesday May 12, 2020 @11:16AM (#23154872)
> I'm confused, a DAG is a model for the relationships between data.

Here they're talking about a DAG as a model for the relationships between computation rather than data.

Many systems that model computation use DAGs. For example it's how many compilers understand your code when you compile it.

Re:Faster and more flexible pipelines with a Directed Acycli... by throwaway894345 on Tuesday May 12, 2020 @11:37AM (#23155116)
> Here they're talking about ... computation rather than data.

The lisp people would like a word...

In seriousness, computation is data, and this is perhaps especially obvious in a CI context where your “computation” is increasingly bash scripts in some string field inside of a YAML file.

Re:Faster and more flexible pipelines with a Directed Acycli... by chrisseaton on Tuesday May 12, 2020 @11:39AM (#23155154)
> The lisp people would like a word...

Why do you feel the need to say things in such a snarky way as this, as if I'm ignorant?

That was the point - that computation can also be represented in a 'data' structure like this. I already gave that exact example - a compiler representing computation as data.

Re:Faster and more flexible pipelines with a Directed Acycli... by qppo on Tuesday May 12, 2020 @11:50AM (#23155346)
I think they're replying that way because you seem to miss my point, I didn't think I needed to say that "the operations are the data" explicitly.

My confusion was not on what they were doing or why they chose to use a DAG, but why they're presenting it this way and why they weren't using a DAG to begin with.

Like I've written task runners in various forms before. They've always been based on DAGs and topological sorting from the beginning. I wouldn't advertise it as such because the choice of model is obvious to me, I'd be more interested in reading about alternatives that provide other benefits.

Re:Faster and more flexible pipelines with a Directed Acycli... by throwaway894345 on Tuesday May 12, 2020 @11:59AM (#23155483)
Whoa. I wasn't trying to be snarky; if anything it was intended to be a humorous jab at how "computation is data" is such a thing in the lisp community.

I agree that your example illustrates the "computation is data" concept; I was just reconciling that with the apparent contradiction ("computation rather than data") earlier in your post.

I wasn't trying to call "gotcha" on you or anything; sometimes I say things that are unclear and I appreciate it when others help me clarify.

Re:Faster and more flexible pipelines with a Directed Acycli... by wolfgang42 on Tuesday May 12, 2020 @11:23AM (#23154947)
They were using a list-of-sets; the CI pipeline is composed of stages which run in series, each of which runs a set of jobs in parallel. The new dependency DAG allows jobs to explicitly declare dependencies on each other and avoid waiting at stage boundaries.
Re:Faster and more flexible pipelines with a Directed Acycli... by tr352 on Tuesday May 12, 2020 @12:40PM (#23156068)
Agreed. First sentence: "A Directed Acyclic Graph will let you run pipeline steps out of order." That's simply false.
Re:Faster and more flexible pipelines with a Directed Acycli... by defen on Tuesday May 12, 2020 @01:59PM (#23157318)
Interpret "order" as "the order in which the steps are defined in your YAML file", and it's simply true.
Re:Faster and more flexible pipelines with a Directed Acycli... by tr352 on Tuesday May 12, 2020 @03:42PM (#23158744)
I'm nitpicking but my point is that a DAG doesn't let you run anything. It might represent something, though, such as a bunch of tasks and a "must be executed before" relation. It's sloppy, imprecise use of language.
Re:Faster and more flexible pipelines with a Directed Acycli... by BorisTheBrave on Wednesday May 13, 2020 @04:55AM (#23164912)
By that logic, no code lets you run anything, it just "represents" a program that actually runs. That's just pedantry.
Re:Faster and more flexible pipelines with a Directed Acycli... by tr352 on Wednesday May 13, 2020 @05:51AM (#23165203)
Code runs and possibly lets you run something (for example in case of an OS or interpreter). I don't see a problem there.
Re:Faster and more flexible pipelines with a Directed Acycli... by gaff33 on Tuesday May 12, 2020 @11:22AM (#23154937)
Historically gitlab had a stages system: "Do all your builds, then all your tests, then all your deployments"
Re:Faster and more flexible pipelines with a Directed Acycli... by detaro on Tuesday May 12, 2020 @11:13AM (#23154848)
Which part of the explanation of what they mean in the article is unclear?
Re:Faster and more flexible pipelines with a Directed Acycli... by corndoge on Tuesday May 12, 2020 @11:18AM (#23154896)
They're sort of coopting the name of a datastructure and using it as if it were a product name which is a bit weird
Re:Faster and more flexible pipelines with a Directed Acycli... by qppo on Tuesday May 12, 2020 @11:45AM (#23155245)
The article is clear on what they're doing. My confusion is to whether this is a feature announcement or a technical blog on the implementation details.