I see it as rather simple. If you're listing off-sets, starting at 0 makes sense, because there'll be something there. But if you're counting elements starting at 1 makes more sense. I don't have 9 fingers offset from my first one, which I call finger zero; I have ten fingers, starting with the first one.
A set with no members is the zero set, once you add one element, you have a set of one. An empty list contains no elements. It does not have a first member, because it's empty. Add an element, and it will have one member, and it will be in the first position. Not in the zeroeth offset from the beginning.
You can the numeric names of letters of a string in an array, but I don't see that as a compelling reason for naming the first letter of a word, the letter which is at the zero-eth offset from where the word is held in memory. At best, it's a low level optimization, at worst it's just a leaky abstraction. Maybe you want to store the length of the word there (like in Pascal), for a different trade-off in terms of what is optimized.
I think you mean Dijkstra, who did say that, because of an argument with mathematicians about indexing from 1.
There's even a section about this on wikipedia: https://en.m.wikipedia.org/wiki/IJ_(digraph) subsection "technical details".
Edit: from Wikipedia[1]: "It used to be common, in particular when writing in capitals, to write Y instead of IJ."
So it's an obsolete practice...
----
0. something along those lines: http://alphabetprintables.org/alphabet_printables_cursive/up...
http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF
Having worked with both, I'm inclined to prefer 0 based addressing in most cases. It's slightly less intuitive, but it generally leads to cleaner code.