This is pretty good advice overall. One small change I suggest is to take it easy on Design Patterns and the like. I’ve seen people in OPs position (general smarts but limited production experience) turn into architecture astronauts and start overengineering everything. It can be useful if you’re working on a legacy codebase and need to understand the jargon that can appear in [possibly overengineered] existing codebases.

I've written software over a decade and I loath the design patterns book.

I would not give it to a beginner as it would corrupt their mind with useless drivel .

It's authors exhibit themselves as morons who celebrate renaming existing computer science concepts while occasionally mixing and matching them.

I would not be this uncharitable towards it if it was not such a famous (and hence harmfull) book.

It's harmfull because software engineering is really hard, and they just add up to the load by trying to have the reader memoize their junk instead of doing something that would actually make them a better programmer. And, if you try to use their concepts while programming - shudders, oh god help you.

I'm not going to iterate over every pattern. Here's an example:

Flywheight? You assholes, just tell memoization for what it is. Why don't you rename existing data structures as well? Good thing those come out of the box, otherwise you would have began the book by renaming array, linked list and dictionary. Maybe you would have called linked list "the chaingang" or something.

You don't simplify things by giving things cuddlier names. You stunt peoples growth that way.

Gang of four book is a malignant offshoot of the practice of attempting to make software engineering better by legalizing and dogmatizing it by decomposition into trivial details that hurt your brain. It's a branch of 'consultancy driven software development' where people attempt to aquire a halo of professionalism by calling things by fancy names and over-complex descriptions (while skipping the practical things with equally complex names but at least those weren't made up by a bunch of idiots).

While I agree that the GoF book is way overrated, I think you're being a bit harsh. Most of the patterns are actually tricks to overcome limitations of statically-typed class-based languages like C++ and Java. They can be useful in these languages but are not necessary with dynamic languages. Take Strategy for instance, with Python or JS, we'd just use a callback but in old Java you couldn't do that, hence the need for a more complicated technique. Creational patterns also mostly fall in this category. Some are so trivial (Template Method, Adapter), that it's hard to understand why they make them such a big deal.

Among the few patterns that are more generally useful and have stood the test of time, we'd find Composite, Command and of course Observer.

Your comment is very interesting. I recently took a course on Design Patterns. I sat squirming during the lectures because I didn't like what was being said, but couldn't put my finger on what exactly I disliked.

What I understand from your comment is that you dislike the Gang of four book because it renames concepts that don't need the cutesy names that they give them. Do you have a problem with the _concept_ of design patterns? Or just the names they are given? Are the concepts themselves sound and worth paying attention to?

I wrote a long rant as a response to another comment above. To quote myself:

"It reads like it was written by a clever, verbose, and 'over-eager novice."

Design patterns are a really usefull concept. The GoF book just totally botches up that concept by dwelving deep in trivial language details while missing the big picture.

Christopher Alexander's "A timeless way of building", and "Notes on the Synthesis of Form" are the books in architecture that prompted a lot of dicussion in software design circles, and from which I presume GoF got their idea of software design patterns.

What are good design patterns in software? IMO they are composed from the programming models exposed in a basic CS book like Aho and Ullman's "Foundations of Computer Science" and further developed in a books like Structure And Intrepretation of Computer Programs.

GoF is an ok anecdotal reference after those, but it really is not suitable as a didactic resource.

Peter Norvig wryly commented that 16 of the 23 pattern are either invisible or non-existent in lisp Lisp[0].

[0] http://www.norvig.com/design-patterns/design-patterns.pdf

I do not like the GOF book. Some of the patterns appears sometimes in my code, but the book has never helped me to program better or helped me to think about my code. The single benefit I got from this book is that it gaves me words to explain my code to other people.

your instinctual reaction against design patterns because you may have realised that design patterns are patches over flaws in the programming language's design - they're a terrible basis to build an architecture on. There's some discussion on this idea at the C2 wiki: http://wiki.c2.com/?AreDesignPatternsMissingLanguageFeatures

Thanks for this post, I remember reading about "Flyweight" and being confused at the time, if it's just memoization that instantly makes sense and is just such a better way of describing things.

I disagree.

Design Patterns are very useful, but they are a tool like anything else.

They're not just a bunch of things with cutesy names, they are the most common and core patterns that we can use in OO.

Yes, they are way overused, particularly in Java, but they are useful.

The book could be summarized however.

Everybody recommends it for no particular reason, and then I read one anonymous post ripping it apart in detail. I’m strongly inclined toward believing the latter and massively discounting most people’s opinion as unfounded, assuming they didn’t read it, and are just trying to convince others that they did. Is this the right way to think about this?

Being charitable, you could say at least part of the former are experiencing confirmation bias - that they enjoyed reading a different take on their existing knowledge - and cannot be arbiters of whether the book imparts anything useful to those without that knowledge.

as a middle ground: my advice would be to learn and internalize design patterns. and then “forget them.” let experience show you when to break the rules. keep things simple.

this level of the craft is subtle, intuitive, and often inaccessible to the conscious intellect.

I agree. I would like to add, doing something like The Rails Tutorial from Michael Hartl would be a great place to start. It takes you through a reasonably real world application including testing and how to think about the constituent parts. A PhD is like being an expert in materials science while lacking in craft-level skills such as running a vertical mill.

To all readers who've never experienced it, working on a large Java (and other) codebases where the original developers completely overused patterns, your life will be hell in my opinion, just as much as a codebase written totally without thought or patterns.

Patterns can be useful but only in moderation and only with reason. They're spice.

A contrived example, but:

https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...

FizzBuzz Enterprise Edition. It's enterprisey!

> folks I saw who did poorly it was because they didn't realize that you could be both the smartest person in the room and the least capable at the same time

Any advice to deal with this kind of people would be much appreciated. I had a hard time trying to convey basic software engineering practices such as not hard-coding file paths, using version control instead of filenames to manage versions, removing dead code, keeping the code at least moderately DRY, more or less sticking to a code convention, etc. They would use all their reasoning power to refute what they would see as meaningless pet-peeves. It was tiring. I tried to provide pointers so they could find out for themselves but they gave it zero interest. At the moment I've given up.

The simple fact the OP is asking this question here means she or he will almost certainly do greatly.

If you have two such people, you might get there by making each of them the sole maintainer of the other's code. They might - might - see the problems when it's being done to them, instead of them doing it to others.

More in general, the problem is that these people have been in an environment where programs were small enough that those practices didn't cause problems. They've moved into an environment where the scale is different, and those practices now cause real pain. Until they experience the pain (rather than just being told about it), they aren't really going to get it.

Sadly it is very much a person by person process. In my experience with folks like this there are people they can "hear" advice from and those that they cannot. One option is to talk to their original PhD adviser (if they are still around) and get them on board. You and the adviser's interests are aligned, you both want the person to succeed and to do so they have to be able to hear your guidance on how to be better at the job. Sometimes they will be able to hear it from someone they have already put in the 'trusted advice' category.

I would also add that lots of new people waste more time doing TDD wrong than they would save doing TDD right. There are lots of things you only really understand until you've done them wrong.

Yeah, a lot of software engineering schools of thought are just based on superstition, but they're all somewhere in the neighbourhood of "productive practices" regardless of how different they are from each other.

If you do

* 100% pairing,

* buttloads of unit tests,

* lots of microservices,

* use an autoformatter,

* in a Node codebase,

to order-of-magnitude approximations you're probably going to be about as effective a team as if you

* always code alone,

* mostly write integration tests,

* write a big old monolith,

* mostly ignore code style (except for egregious cases),

* in C.

These variables do make a difference, but those differences are context-dependent and can be swamped by other factors. Use version control. Do at least some testing and some code review. Make sure engineers can actually run their code before submitting it (on their laptops or in special staging environments or whatever, it doesn't matter much.) Help coworkers when they're stuck, and ask for help when you're stuck. And try to hire Harrison Bergeron onto your team :-).

When programs are simple, usually time to write code is dominated by finding the requirements. As programs become more complex, the time to write code becomes more and more dominated by understanding and working with the existing code.

In code bases you have written yourself, often you remember enough about it that you can mitigate the effects of complexity. Possibly you already had a plan for how to implement some functionality. Or maybe you know how to avoid certain problems.

As programs become more complex, you will find that "legacy team members" are disproportionately productive. Learning the code base becomes more important than learning the problem domain.

Teams that expend effort on maintaining a status quo where the exploring the problem domain dominates the cost of development (even with new developers), will usually lag behind teams that do not. Mostly, because it is difficult to do.

Some of the practices you mention can be used to help in this, but for a "legacy team member" it might not be attractive. If you have a more typical situation, knowing the code base better means you are much more productive and therefore more valuable. This secures a senior position on the team. Turnover is usually high, so most people leave before it gets to the point where even "legacy team members" become unproductive. When you go to a new shop, you can always complain about the "crappy" code and suggest a complete rewrite (at which point, you secure your senior status again).

While your list does not necessarily comprise what I think of a "productive practices", I think it is incorrect to imply that "productive practices" can not have a profound impact on the outcome of a project. The productive practices that work best for each team will likely be different. It's important to pick practices that work for your team and produce code that is simple to read and extend. Not thinking about it is a good way to ensure that you will end up with a complex system.

Whoa I just read about Harrison Bergeron on wiki [0]. That was an interesting read. I'm sure the full story would help capture a lot more of Harrison's spirit and qualities

[0]: https://en.wikipedia.org/wiki/Harrison_Bergeron

On this vein "Ian Cooper - TDD where did it all go wrong?"

https://www.youtube.com/watch?v=EZ05e7EMOLM

I wish I had watched that video two years ago! I learned all of that the hard way, and it took lots of failures.

I want to second about the "Code Complete" book. IMO, it provides a much more practical and hands on approach than most of the other books, and does this to a great depth. In my view, if I'd recommend only one book on good software engineering practice, it would probably be that one.

On a related note, I just this week stumbled upon a post / essay, which was probably the best piece of writing I've read in computer-aided engineering, and working in a disciplined manner. I'd recommend reading it right now, as the very first thing you do: https://queue.acm.org/detail.cfm?id=3197520

> read "The Design of the UNIX Operating System", read "TCP/IP Illustrated Volume 1"

What? Those are unnecessary details as far as software engineering is concerned.

I think you are interpreting "go read" as in go learn these by rote. He's not saying go memorize the details of these Unix and TCP books. He's saying go read them and absorb their gestalt approaches to decomposing complex systems so you can think about your own problems that way.

To this list, I would also suggest compiler construction or database systems as different topics exhibiting the same thing.

A more basic challenge for a scientific programmer might be absorbing the mindset of tool building rather than problem solving. It's a different mental perspective to think of your code as a product for other developers rather than as a means to producing a data artifact.

I would be interested to know why you think of them as unnecessary details.

What I like about them are that they describe large complex systems that have to work with a bunch of people plugging in their own implementation of the parts. For me, both books give a good sense of what you need to do to enable the product you are working on to evolve successfully, and highlight areas where a lack of flexibility impacted that evolution.

So often in programming you look at your code and have to evaluate not only what features you are delivering but also what paths you are closing off with your design. In my experience the folks who excel at programming hold both the model of the system in their mind while they execute some part of it.

I think both books give a flavor of that.