Started by building my own build system, monitoring solutions, taking at stab orchestration problems, graph databases etc.
The problem statement would be super simple like how do I install my application on 100 servers? And I will start off with bash, tar, ssh and work my way up.
The goal isn't too finish. You want to discover the problems first hand, decide upfront how you would evaluate an acceptable solution, attempt a naive solution, and then look at an open source system, note the family of algorithms + data structures then loop back and move onto the next set of problems in the domain.
Ideally the system you're building is distributed.
Mostly I work from the outside going in and counterintuitively I start with the two non-functional requirements, as questions: how fast do you want it to go, secondly how badly can the system fail. I find that I'm peculiar in this approach.
This won't work for everyone. It has for me because it's allowed me to build a handy library of interrogative and evaluative heuristics and develop strong understanding system architectures and design
- There is general systems theory [0] and within that systems engineering[1] where you will find software systems [2]
- OP is most likely talking about how to use of general theory of systems in different domains and not specifically software systems.
- If one considers JQ, SED and "One input stream, one output stream" and then compares that to a model of a system using a CLD (Causal Loop Diagram). A CLD is effectively "An input, followed by an output"
- Another word that could be used for "input stream" would be 'flow' from system dynamics [3]
- One could apply general systems theory (the abstract concept) to just about anything
[0] https://en.wikipedia.org/wiki/Systems_theory
[1] https://en.wikipedia.org/wiki/Systems_engineering
There are a _lot_ of systems out there that tie together various components with tools that behave a lot like jq (or literally are jq...)
There is nothing unique about jq in this regard, and there is nothing particularly “systems thinking” about any of them. You can and people have used them for decades without doing any special “systems thinking”.
Also a good example of loose coupling that allows for good thinking of systems.
In school we watched the 1990 movie [MindWalk](http://www.imdb.com/title/tt0100151/). You can watch it on YouTube https://youtu.be/Uec1CX-6A38
Does there have to be?
It's simply an example of a good systems pattern/approach.
I'd also substitute Ansible with Nix/NixOS because the conceptual takeaways are much greater there.
Well, except for reactive programming
If you are a software engineer:
Build a simple database, a simple interpreter, and a simple web application (backend and frontend). Read about “distributed systems”, “devops”, “Conway’s Game Of Life”, and “fractals”. Read about these technologies/techniques (in whatever order is comfortable), how they work, their pros and cons, why they were created, how they compare to alternatives: “TCP”, “Open Sound Control”, “Plan9”, “REST”, “Lisp”, “Erlang”, “Smalltalk”, “Forth”, “Datomic”, “event sourcing”, “reactive programming”, “communicating sequential processes”, “APL”/“J”/“K”, “Ansible”, “jq”, “graph databases”, “Apache Kafka”
The idea is not to become an expert in any of these, but to digest the wisdom that lies in their design and surrounding literature. Each was created after lots of careful thought by brilliant people about how to build sustainable systems.