This very much! I wasted so much time trying to "memorize" the fretboard. It comes naturally by practicing music. Almost all of it began from knowing just the low E string and applying first octave patterns, then other intervals as I understood what they were. Without musical context, it's not really all that useful to know how to instantly name a note on a fret. Maybe finding octaves quickly, given a starting note, could be useful in the time-based game realm.