It true that there's an argument that says you often get more dividends for your effort wrestling with larger architecture and design than about, e.g., how easily you can write (or reuse) a binary tree implementation. A library won't help you implement the binary tree implementation for accessing a filesystem's on-disk data structures, for example. The value of some of the most beneficial use cases is often precluded to generic implementations. I think that reality played into why they discounted the value of generics, making them a low priority, but it's subtle and complex.
Generics (and meta programming more generally) cut both ways in terms of cost and benefit, at least when you think more holistically, and especially when you think about the complexity of the language and language implementation. It's a complex topic. There's no right answer, at least no right answer without a careful accounting of the context--the problems you're trying to solve and which aspects are the most critical or the most difficult to solve. That's the point.
Another way to think of it is that Go's structural typing (loosely speaking, duck typing) had much more bang-for-the-buck than generics, both abstractly and concretely. If you came to that conclusion early on, you may design and implement the language in a way that makes generics even more difficult to fit into the language. The original decision could have been perfectly legitimate, but over time it might seem like you simply didn't appreciate generics at all, especially when criticisms take the benefits of the existing design for granted without thinking about path dependency, etc.
FWIW, I've never written a single line of Go code.