I feel like declarative is some kind of technical manual meme for the last few years. I don't know at this point how often I've read that something is better because it's modern and declarative. Why it's easy to understand because it's declarative, etc.

There's a similar thing with "functional". In that case I at least understand the intuition why it's better, which is because if I/O behavior is constant and you don't have to think of an internal state, like in OOP. It is indeed much easier to divide and conquer when you can understand the parts of the whole individually. And as soon as you understand the parts you can piece them together.

With "declarative" I still have to find out what's the intuition WHY it's supposedly better, until then it's kind of like the 90s, where the washing machine was "better" because it had "fuzzy logic".

I feel most comments here miss the point of declerative systems - resillience.

You write a script to stand up an Application on a clean system, a declarative system does the same. But what if current state is not a blank slate? A declarative system is meant to be smart enough to achieve target state even if machine crahsed half way through the setup last time, or has a different config already running.

Writing a shell script that addresses all edgecases is hard. Now, declarative systems do fail, but that's the promice.

The nuance is subtle but important.

Let's say you stop tracking that file. In Chef, the file will simply remain in its last known state, because Chef forgot about it. Likewise, if you do the same process on other systems, files will be left in various states - causing configuration drift. This is what a declarative tool is meant to solve.

A proper declarative tool would _delete_ the file, because it's no longer tracked, to ensure that the system is in the state the user expects.

Declarative is better because you write what you want to _have_ not what you want to change, which means you need implicit knowledge of the state of the system, and leave the complexities of figuring out the in between to the tool you're using.

And then you're in a world of pain, where you have to not only know how to do what you want, but then also how the system will map from declarative->imperative, and then how to massage your declarative input so that it does the right imperative thing. And your teammates too. And you've all got to think about it every time you change the code or upgrade.

I've run into this problem in nearly every declarative system I've ever used (examples: SQL, Prometheus, infrastructure-as-code frameworks, dependency injection frameworks, ORMs, k8s, and so on).

Neither approach is "better". They have tradeoffs.

But this is the case with opinionated imperative interfaces as well. It's irrelevant to the declarative vs imperative issue. This is a fundamental aspect of abstractions. There will always be things you can't do unless you're writing assembly on bare metal. And even then, assembly is an abstraction over microcode, and what the actual hardware is doing.

The boundaries of an imperative api are atomic and easily well-defined. When something goes wrong, it is possible to look at the methods individually and figure out exactly where expectations and reality diverged.

With a declarative system you just put out everything there as one big chunk of state and hope the right thing happens. When the wrong thing happens, it's usually not clear if the problem is a bug with the engine or your understanding of it, and also not clear how to go about determining where the problem is.

To be clear, I am not totally against declarative systems, but I would be hesitant to use any declarative system if it isn't mature and well-tested, documented with priority, and in a space where the use cases are predictable.

The vast majority of time, you just want your SQL query to give you back the right rows and don't care how the DB engine manages it. In the rare case where it takes too long, then you want to be able to dig into the query plan, change the indexes, maybe even deploy a custom function written in a different language in extreme cases.

Can you give an example of this for k8s?

Deployment.spec.strategy is a common purely "how" configuration switch.

Explicit PodDisruptionBudget is less common but exists only to configure a "how". A smarter (e.g. layer 6+ aware) system should be able to derive this for me.

You might respond "well, that's what an operator is for" and in some sense you're correct, but every operator I've read is a mess of the most imperative code I've ever seen; and then that leaks out as soon as the operator's defaults don't work well for your case.

That's a bit like sating car is only better than a horse if the engine works. The tool has one job,

They do ofcourse fail regularly, I feel that devops tools are 10 years behind our development tooling

Also, you really do need to know how things are going to change when they're changed. By default when you update your deployments in k8s, you're probably going to have a brief amount of downtime. Argo rollouts and such let you define this declaratively, in a way that avoids downtime, but again, more complexity.

Many of our traditional operators / SysDE greatly prefer reams of declarative configuration. They like to be able to look at a system configuration and understand it in totality. Even if there are abstractions. These folks tend to run other people's software, be great at reading manuals, and debugging systems they can't see code for. The reasoning for a given setup is usually in a wiki or is common knowledge (to some extent) in their heads and the organization expertise of running that software for years.

Most of our engineers (SDE) hate declarative as it's not DRY, really boring, and doesn't encode the intent as you said. Code based solutions let a developer (who groks code) understand some of the rules behind WHY (to some extent), and developers tend to be lazy and want to move that knowledge out of their head and into computers so they don't have to deal with it.

Both sets of people can be right.

This is actually the next generation of devops tools... CDK, Pulumi, etc.

Some of our ops folks still prefer pure declarative. Like down to the explicit yaml and knowing exactly what AWS flags are ticked in what way. They're also much less comfortable with code.

It's not an argument it's a preference, it's how their brains think and their comfort zone. They would literally rather see 1.5 MB of zipped yaml then read through CDK libraries. They literally look at the generated CFN over the cdk.

Declarative means “some system reads and parses the config, can do clever things to change what happens in different circumstances, show you a preview of what happens, track chances of what the config has been in a straightforward way, audit the config for consistency across several systems or for adherence to selected principles, and do other smart things”. These are possible because the config is a data structure now. You can do things with a data structure you can’t do with a shell script. The script is too opaque and our computers are not yet smart enough to analyze it in these ways.

There are great advantages to the simplicity of imperative configuration scripts, which is why shell scripts will not soon die. There is some peril (shell scripting in particular is full of tricksy traps; alternative languages may mitigate this.) There is great power and some peril in imperative config. It’s a choice.

>, why it's implicitly assumed to be better is what eludes me.

I think the common generalization of why declarative is almost always better than imperative is simply this:

  strlen(declarative_statement) < strlen(imperative_statements)

So a more concrete example is:

  strlen("SELECT * FROM customers where state='CA'") < strlen("open binary db file, seek to disk sector xyz, btree search index id, allocate memory buffer for cursor results, etc, etc")

  strlen("We want to increase sales of suntan lotion.") < strlen("SELECT \* FROM customers where state='CA' or state='FL'; select \* from products where category_tag = 'suntan_lotion'; [...more SQL instructions...]")

> [in functional programming] you don't have to think of an internal state, like in OOP.

You still have to but less than with OOP languages. For example I've got an Elixir GenServer that uploads files to Google Storage and keeps retrying of something goes wrong. The internal state of the server is the list of files to upload. Files get added to the list when the server receives a message with a file and removed when the file is uploaded. Tests have to deal with the internal state, as in OOP.

This kind of stateful "functions" are implemented by recursively passing the state to the same function. Tail call optimization helps not overflowing the stack.

Of course we have state only where it matters so the servers are few and sparse. But sometimes we have to pass large data structures between functions. They would be the internal state of an object in OOP. Check Ecto's Multi for transactions.

For instance, you can declare that you need two X (e.g. containers, disks, …) and pass that configuration to the framework. If the framework sees you currently have 1 X in production, it allocates another. If there are three, the framework kills one. And if there are 2, it’s a no-op. The beauty is that if you express what you need declaratively, you’re shielded from deciding the state transition.

Imperative: open the top drawer, then take out a knife, then take out a board, then open the basket, take out a potato, slice the potato with the knife 4 times in parallel, then turn it 90 degrees and do it again.

Declarative: make some diced potatoes however you like, whenever you like. Have it ready for when the frying pan is hot.

Imperative lets you be very precise about what to do and when to do it, what resources you need, etc. Declarative lets the system decide things like execution plans (and thus algorithm and resources) so you can spend your time specifying the target state rather than all the steps in getting there.

In my experience, describing the end result in a way the program understands is sometimes much harder than telling the program "do this".

Don't get me wrong I think the longevity of SQL shows that declarative can be the right way. However I think there's a logical leap with declarative = better.

It’s also easier to understand the current state of the system. In an imperative style, you need to understand every operation that has been performed to know what the current state is. In a declarative system, you just need to look at the most recent declaration.

In short, declarative is more scalable and understandable.

Another way to look at your question is to ask what is better about the imperative style? Having used both, I can’t think of any advantages. Maybe you could say you have more control with imperative, but is that control really necessary? In my experience, it is not. The additional control just makes things more complex.

And of course you can make a overly complex, hard to understand system using a declarative paradigm. But the simplest declarative system can be simpler than the simplest imperative system for achieving the same configuration.

If you don't have to debug your system in the large (why is my outcome not what I expect)

If your operators are written correctly and you don't have to debug your system in the small

Your underlying assumption is that it’s easier to write correct imperative configuration than it is to write correct declarative configuration. In my experience, this is not the case. Consider a basic task like provisioning a fleet of hosts and deploying some code to the fleet. In an imperative approach, you need two distinct steps. One for provisioning, and one for deploying. Both operations are multi-step processes that are required to be idempotent because the number of possible failures between the start of provisioning and the end of deployment is quite large. It’s not easy to write a system that does this, as there are a lot of steps that need to be enumerated via imperative scripts for both provisioning and deployment, and it’s even harder to do in a failure tolerant way.

Compare the above to a Kubernetes deployment. The entire configuration is 30 lines of yaml. There is no providing step. Kubernetes will make sure the resources you need are provisioned and it will do so in a failure tolerant way.

By contrast, bugs in imperative code can manifest simply because you happened to write it in such a way that relied on some assumption that no longer holds and your imperative code does something it should not do because its preconditions are invalid.

Declarative approaches are generally more robust because they by necessity tend to be designed to handle more diverse initial states and still reach the correct result.

[0] iirc this was not a bug in their kubernetes operator (or maybe it was, I don't recall exactly) but a situation where they didn't understand why their containers were exhibiting the anti affinity properties they were expecting out of their declarations.

Anyone who's had to do an EXPLAIN PLAN or equivalent to figure out why a SQL query is suddenly taking 40x the time to execute and return results can attest to this I think.

It's really just a preference for convenience over control.

If it can achieve whatever you want to be done, is it not more logical, natural, and human to just describe what you want the end result to be? The answer is an unequivocal "yes".

You've already received the answer. Here in this thread, but also, I'm guessing a great many times. You just refuse to accept it. As I've said, "a 'you' problem".

Whoever came up with that ... XSLT or Prolog are declarative yet I challenge both of them to be hard to understand

I have a feeling XSLT would be better known (i.e. be able to justify investment into learning) if not for json/yaml popularity. Though XML is not particularly human readable anyway. The next logical step after https://json-schema.org/ would be JSLT I imagine.

On a related note, Maven is a great example of a declarative build system. Both Gradle and SBT were gigantic steps back.

Declarative is generally better because you move complexity out of your code to someone else's code. In other words, it reduces the LOC you write and have to maintain.

For example, say you are writing some code to query Postgres and aggregate a column. The imperative approach would be to query Postgres and do the aggregation yourself. The declarative approach would be to query Postgres and ask it to do the aggregation via GROUP BY.

Aside from reducing complexity, it's also usually more efficient to take a declarative approach. Dozens if not hundreds of people have been working on Postgres for almost 20 years now. The chances of your own aggregation algorithm being more efficient than Postgres's is practically zero.

These solutions don't necessarily have anything to do with the concepts as you name them.

Why not? A "GROUP BY" syntax is -- relatively speaking -- more declarative than handwritten imperative code looping through a cursor and dynamically adding to a keys-values data structure to accomplish the same idea.

I wasn't talking about strict definitions of "declarative programming" vs "imperative programming" to satisfy language lawyers.

It was a continuation of the subthread discussion started by carlmr asking why declarative is better: https://news.ycombinator.com/item?id=30098793

So it was just talking about aspects of "declarative" in general...

The idea is at a small scale, one can compare an imperative loop construct to the more "declarative" syntax that accomplishes the same task. Yes, the "more declarative" syntax will be a higher-level abstraction.

Another example of the micro-instead-of-macro type of discussion was C# when it added LINQ (declarative quasi-SQL) and async/await (declarative futures). The language designers and tutorials used the word "declarative" to describe those new language features even though C#-as-a-whole is not formally categorized as a declarative language. Does using LINQ turn C# into a declarative language? No, but nobody claimed that.

But you are right that it is higher level in that it abstracts away the logic behind GROUP BY. In this sense, all declarative programming is higher level.

So, you're not talking about the same thing as OP, because one of the points being made here is that Kubernetes intentionally tries to get rid of human operators for day-to-day decision making. If you have a machine making operations decisions instead of humans, then a "bash-like shell" is worse than watch/subscribe mechanisms with full type safety, plus standard conventions for reporting events and communicating status.

It is useful to have both the fitted arrangements and operator interaction.

In addition to giving operators power to make changes in production, the command line is useful for development. Devs can develop in a one-host grid and interact with it in a repl-like way. Contrast this to traditional workflow where the deployment environment is quite different to the development environment. This encourages devs to work with the grain of the platform and discourages the "it works on my machine" dynamic.

You could offer something like shell scripting on that, but my system did not and there are better options. For example: have the resource manager expose a command that offers a DSL for making system-wide changes.

Like... Kubernetes? :)

The Kubernetes operator is software that continually compares what-is to what-should-be and makes a change that changes the state of the world to be closer to what-should-be. You specify what-should-be and (if all is well with the world) the system morphs itself into that in a more or less expeditious way.

The original article described this pretty well.

Wouldn't a true SSI need to include some version of software-implemented distributed shared memory, which in turn would need to be achieved by involving the kernel? Linux now has process checkpoint and restore which will actually work seamlessly if all resources are properly namespaced ("containerized"), but not yet DSM, so far as I can tell.

(The only alternative I can think of would involve hacking around the debug infrastructure to "attach" to other processes and somehow handle VM faults in userspace, which sounds positively terrible.)

Yep. K8s is a hell spawn of ITIL (a decades old IBM's take on BigCo/government IT infrastructure management). Google has been practicing ITIL, Borg comes out as a result, which takes K8s shape outside of Google. That is one of the main reasons why management likes K8s - ITIL was created by managers for managers. The approaches like SSI for example shift the balance toward developers/engineering and thus instinctively resisted by managers.

Having been a member of the team that built Borg, I can assure you that those two things were unrelated. I can't say for sure whether the initial idea/impetus for Borg came from the management or the developer side, but once the project was started it was very much engineer-driven, and pretty organically so. The Infrastructure team at Google at the time (of which Borg was a part, along with several other related cloud projects) had a single manager for what grew to eventually 100+ engineers (mostly senior/staff level, and highly independent) before Google started creating/hiring managers more aggressively, so we were pretty much self-managing. The design and implementation of Borg was horizontal, both within the team, and coordinating with engineers (developers and operations teams) that would actually need to use the system. I'd say it was at least three years in before management really got involved in design decisions, by which time Borg was running most of Google's production services

It was a bit of a Wild West, but compared to some later major from-scratch design/development projects I've been involved in since, it was a Belle Époque.

K8S has controllers that are given a configuration and work out how to deploy that, taking into account current state. ie, it's functionally taking the state of the system now, S0 and applying functional modifications to return S1. The issue is that the modifications aren't "pure" in a functional sense, because there's a temporal aspect that means there can be difficulties in achieving an equilibrium to get to S1.