>On an infinite tape, there is no "first byte", only "a byte placed where the tape already was when you started" (i.e. at offset=0.)

Maybe, but human brain likes things starting at 1.