I feel like if you walk into a job interview knowing the corner-cases of a two-phase commit and being able to solve a problem using Lamport timestamps, you're probably in the top 90th percentile of dev applicants.
Maybe ~99th.
The author has been developing software for 20 years. He is likely a fine applicant for a significant number of software dev positions, since he can learn and apply many different technologies very quickly, from what he has said. And also, from what he has said... he doesn't come close to what you described.
I've been developing software professionally for five years. I've been programming C/C++ for 10. (I'm 23; my passion has been for gamedev.) And I don't come close to what you described.
If I devoted myself to learning what you just described, I could probably achieve a thorough understanding (deep knowledge, an important distinction from superficial knowledge) inside a month. But at the end of that, it seems doubtful I'd be much closer to accomplishing the author's stated goal... I would only know two essentially random cornercases.
All of that said, thank you (and SpikeGronim) for mentioning Lamport timestamps; time to go a-wikipedia'n.
If you want more Lamport goodies: "Paxos Made Simple" (distributed transactions done right): http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.69....
"The Byzantine Generals Problem" (harshest failure model known and how to cope with it): http://research.microsoft.com/en-us/um/people/lamport/pubs/p...
An interview where he talks about his approach to systems, particularly formal reasoning and specification: http://www.budiu.info/blog/2007/05/03/an-interview-with-lesl...
His publication list - http://research.microsoft.com/en-us/um/people/lamport/pubs/p...
It won't take you a month to learn these things, but with commit protocols in particular, you have to implement and test them to really grok them.
Exactly, that along with being in the company of people who know more than you. Studying and experience. One learns things like this incrementally, like anything else, and nothing is better than a good teacher.
I am asking a question on continuous deployment here. http://news.ycombinator.com/item?id=2288382
Was wondering if you can spare some advice for me?
Thank you.
For distributed systems there are two main things to learn from: good papers and good deployed systems. A researcher named Leslie Lamport invented a number of key ideas such as Lamport timestamps and Byzantine failure models. Some other basic ideas include quorums for replicated data storage and the linearizability consistency model. Google has published some good papers about their systems like MapReduce, BigTable, Dapper, and Percolator. Amazon's Dynamo paper was very influential. The Facebook engineering "notes" blog also has good content. Netflix has been blogging about their move to AWS.
Every software engineer needs to manage complexity, but there are some kinds of complexity that only show up in big systems. First, your system's modules wil be running on many different machines. The most important advice I can give is to have your modules separated by very simple APIs. Joshua Bloch has written a great presentation on how to do that. Think about what happens when you do a rolling upgrade of a 1,000 node system. It might take days to complete. All the systems have to interoperate correctly during the upgrade. The fewer, simpler interactions between components the better.
The best advice I know of about operating a big distributed system is this paper[1] by James Hamilton. I won't repeat its contents, but I can tell you that every time that we didn't follow its guidelines we ended up regretting it. The other important thing is to get really good with the Unix command line. You'll need to run ad-hoc commands on many machines, slice and dice log files, etc.
How did I learn these skills? The usual mix of how people learn anything - independent study, school, and building both experimental and production systems.
1. http://www.usenix.org/event/lisa07/tech/full_papers/hamilton...