He's not complaining about using vectorization in code. The problem is papers and even explanations targeted at non-experts, often use obfuscated math in place of clear explanations. I've complained about this before here: https://news.ycombinator.com/item?id=13953530

Mathematical notation is basically a programming language. A programming language with weird symbols you can't type to search for, single letter variable names for everything, and no comments. And it's written by programmers that are obsessed with fitting everything into a simple line and making it as small as possible, no matter how difficult it is to read. Any programmer understands this is incredibly bad practice. And even if parse every step and perfectly follow what the code is doing, without explanation, it's pretty difficult to figure out why.

> Mathematical notation is basically a programming language.

A very bad one that can only be executed by brains with the requisite existing historical knowledge; in fact it's more like bad pseudo-code that lacks the explicitness necessary to translate into actual instructions. It's basically condensed jargon intended for the already converted.

It'd probably be vastly easier to teach math with an actual programming language than with traditional notation. Scheme would be ideal for this.

OK, I see what you're saying. I think you have the same issue with "real" programming languages too. If you compare some very concise Clojure or Scala code with the equivalent in Java, it can be quite hard to understand if you're not very familiar with the language. But I wouldn't necessarily say it's "incredibly bad practice". A Scala programmer can write concise and elegant code that to another Scala programmer is actually faster to understand because of that conciseness. Whereas the same code written with for loops and class method calls and all the boilerplate in Java would take more studying to filter out the low level constructions.

It's about the level of abstraction. And yeah if you don't understand the notation or syntax at the level of abstraction you're studying, it will be very hard.

(FWIW I find Scala code quite hard to understand sometimes, but I also find the more I know about the language, the more comprehensible it gets).

It's not necessarily the conciseness that's a problem. Using foreach instead of a full for loop is one thing. What I'm complaining about is code in place of an explanation. E.g. imagine coming across some nasty piece of code like this: https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv...

It doesn't matter how familiar you are with the language. Without an explanation of what the hell is going on, just looking at the code is useless.

Now we're talking about documentation. You are correct, no code is conpletely self documenting. But that Quake code is very low level, the opposite of what I think the grandparent doesn't like (very high level abstract notation)

I think OP means vector notation in the research papers, not using vector operations and representations at the code level.

That said, it can be useful for the beginner to implement a basic NN library from the ground up, to understand how the vector processing works, as well as what is going on step-wise with backprop and such.

Once that is fully understood, the next step of utilizing true vector processing libraries can be taken, and so on - eventually culminating in using and understanding libraries like TensorFlow.

Having the background of the lower levels gives you an appreciation and even insights when you transition to higher level frameworks.

That's just my opinion, though.

This has nothing to do with understanding backpropagation (which (correct me if i m wrong) is really the core of DL). In fact in the old days backpropagation was all about "propagating the deltas" and nothing about vectorizing.