News for hackers. Stuff that matters.
Re:Ask HN: Any recommended resources to develop system think... by notduncansmith on Thursday March 29, 2018 @03:09AM (#16705125)
Rich Hickey has some great talks on YouTube.

If you are a software engineer:

Build a simple database, a simple interpreter, and a simple web application (backend and frontend). Read about “distributed systems”, “devops”, “Conway’s Game Of Life”, and “fractals”. Read about these technologies/techniques (in whatever order is comfortable), how they work, their pros and cons, why they were created, how they compare to alternatives: “TCP”, “Open Sound Control”, “Plan9”, “REST”, “Lisp”, “Erlang”, “Smalltalk”, “Forth”, “Datomic”, “event sourcing”, “reactive programming”, “communicating sequential processes”, “APL”/“J”/“K”, “Ansible”, “jq”, “graph databases”, “Apache Kafka”

The idea is not to become an expert in any of these, but to digest the wisdom that lies in their design and surrounding literature. Each was created after lots of careful thought by brilliant people about how to build sustainable systems.

Re:Ask HN: Any recommended resources to develop system think... by thinkersilver on Thursday March 29, 2018 @09:59AM (#16707095)
This is how it worked out for me. I tried to build the system myself and note the abstractions, edge case. It didn't make me an expert in all things but gave me the right questions to ask. As you do enough of these your expertise will crystallise in a particular domain.

Started by building my own build system, monitoring solutions, taking at stab orchestration problems, graph databases etc.

The problem statement would be super simple like how do I install my application on 100 servers? And I will start off with bash, tar, ssh and work my way up.

The goal isn't too finish. You want to discover the problems first hand, decide upfront how you would evaluate an acceptable solution, attempt a naive solution, and then look at an open source system, note the family of algorithms + data structures then loop back and move onto the next set of problems in the domain.

Ideally the system you're building is distributed.

Mostly I work from the outside going in and counterintuitively I start with the two non-functional requirements, as questions: how fast do you want it to go, secondly how badly can the system fail. I find that I'm peculiar in this approach.

This won't work for everyone. It has for me because it's allowed me to build a handy library of interrogative and evaluative heuristics and develop strong understanding system architectures and design

Re:Ask HN: Any recommended resources to develop system think... by bauerd on Thursday March 29, 2018 @04:22AM (#16705423)
Huh? I currently work with some of the technologies you mention, but fail to see the connection to systems theory. Can you elaborate why e.g. jq or Erlang are relevant here?
Re:Ask HN: Any recommended resources to develop system think... by ismail on Thursday March 29, 2018 @11:35AM (#16707887)
- There seems to be confusion here.

- There is general systems theory [0] and within that systems engineering[1] where you will find software systems [2]

- OP is most likely talking about how to use of general theory of systems in different domains and not specifically software systems.

- If one considers JQ, SED and "One input stream, one output stream" and then compares that to a model of a system using a CLD (Causal Loop Diagram). A CLD is effectively "An input, followed by an output"

- Another word that could be used for "input stream" would be 'flow' from system dynamics [3]

- One could apply general systems theory (the abstract concept) to just about anything

[0] https://en.wikipedia.org/wiki/Systems_theory

[1] https://en.wikipedia.org/wiki/Systems_engineering

[2] https://en.wikipedia.org/wiki/Software_engineering

[3] https://en.wikipedia.org/wiki/System_dynamics

Re:Ask HN: Any recommended resources to develop system think... by amelius on Thursday March 29, 2018 @04:56PM (#16710956)
There's also linear system theory, which control theory is related to.
Re:Ask HN: Any recommended resources to develop system think... by bordercases on Thursday March 29, 2018 @04:25AM (#16705440)
Most of those technologies are known for either how they handle complexity, or the coordination between asynchronous actors with shared resources
Re:Ask HN: Any recommended resources to develop system think... by gaius on Thursday March 29, 2018 @04:57AM (#16705553)
Uhhh what? jq is basically an updated sed. One input stream, one output stream, no awareness of anything else
Re:Ask HN: Any recommended resources to develop system think... by JshWright on Thursday March 29, 2018 @05:58AM (#16705794)
You don't see how a simple input/output streaming model would be helpful in understanding systems theory and design?

There are a _lot_ of systems out there that tie together various components with tools that behave a lot like jq (or literally are jq...)

Re:Ask HN: Any recommended resources to develop system think... by gaius on Thursday March 29, 2018 @06:12AM (#16705852)
Yes this is how every Unix command line utility works.

There is nothing unique about jq in this regard, and there is nothing particularly “systems thinking” about any of them. You can and people have used them for decades without doing any special “systems thinking”.

Re:Ask HN: Any recommended resources to develop system think... by prepend on Thursday March 29, 2018 @08:23AM (#16706393)
The Unix model is really useful in my systems thinking precisely because thousands of components all use the stream in/out approach. This made then reliable within a system to be used together in many different combinations and interactions.

Also a good example of loose coupling that allows for good thinking of systems.

In school we watched the 1990 movie [MindWalk](http://www.imdb.com/title/tt0100151/). You can watch it on YouTube https://youtu.be/Uec1CX-6A38

Re:Ask HN: Any recommended resources to develop system think... by sheepmullet on Thursday March 29, 2018 @07:31PM (#16712202)
> There is nothing unique about jq in this regard

Does there have to be?

It's simply an example of a good systems pattern/approach.

Re:Ask HN: Any recommended resources to develop system think... by bordercases on Thursday March 29, 2018 @10:55PM (#16713349)
I took your "e.g." to mean inclusive of jq and Erlang, but extensive to the entire list. What I said applies to Erlang, OSes, and LISP, but maybe not to jq (I wouldn't know).
Re:Ask HN: Any recommended resources to develop system think... by joncampbelldev on Thursday March 29, 2018 @05:51AM (#16705768)
Perhaps they meant it as a good example of the type of component to build systems out of, simple input->output. Unsure though.
Re:Ask HN: Any recommended resources to develop system think... by tazjin on Thursday March 29, 2018 @09:24AM (#16706786)
Missing the thing that has had the single biggest impact on how I personally think about programming: Learning a properly typed language (e.g. Haskell).

I'd also substitute Ansible with Nix/NixOS because the conceptual takeaways are much greater there.

Re:Ask HN: Any recommended resources to develop system think... by blueprint on Thursday March 29, 2018 @09:02AM (#16706615)
> how to build sustainable systems

Well, except for reactive programming

Re:Ask HN: Any recommended resources to develop system think... by ioddly on Thursday March 29, 2018 @04:29AM (#16705455)
Where does one get started building a simple database?
Re:Ask HN: Any recommended resources to develop system think... by weavie on Thursday March 29, 2018 @05:00AM (#16705568)
I found Martin Kleppmans book - Designing Data-Intensive Applications (http://martin.kleppmann.com/) to be a great primer.
Re:Ask HN: Any recommended resources to develop system think... by goostavos on Thursday March 29, 2018 @01:18PM (#16708864)
An absolutely _fantastic_ book. It was also one of the scariest books I've read recently. Made me realize just how much I don't know (and don't know that I don't know!) about the failure modes of distributed systems.
Re:Ask HN: Any recommended resources to develop system think... by gothy on Friday March 30, 2018 @10:38AM (#16716424)
This is a must read book even if you're not building a database :)
Re:Ask HN: Any recommended resources to develop system think... by tayo42 on Thursday March 29, 2018 @11:36AM (#16707900)
Something like a kv store with a btree or sstable. https://github.com/antirez/otree simple example I think
Re:Ask HN: Any recommended resources to develop system think... by bordercases on Thursday March 29, 2018 @04:53AM (#16705544)
Can you develop a file format plus API that is ACID compliant in C?
Re:Ask HN: Any recommended resources to develop system think... by elvinyung on Thursday March 29, 2018 @06:41AM (#16705944)
Careful with that word ("ACID"), it's ridiculously easy to get screwed by the filesystem arbitrarily lying about things like the order of system calls: https://queue.acm.org/detail.cfm?id=2801719
Re:Ask HN: Any recommended resources to develop system think... by jmalicki on Thursday March 29, 2018 @11:07AM (#16707637)
The file system does not lie, it just does not provide guarantees of ordering.