Others just search for known text, which IMO is a habit I need to get into. I "got" hjkl after I disabled the cursor keys for a day, but I never followed up and disabled hjkl (or maybe even w and b!?) to get super efficient.
Also with the right setting(I think incsearch) Vim immediately jumps to the first result as you type, so I often use search to quickly peek at some code and then jump back with Esc when it doesn't warrant splitting the buffer.
But I do still struggle a bit with moving within the line - w and b tend to "get caught" in long series of different symbol categories that Vim will all see as separate words/tokens. At least I got my setup with the "camelcasemotion" plugin working, which is a great help with long identifier names.
The only time I find using :number is when I am exploring a codebase elsewhere, e.g. using GitHub’s web interface, and want to jump to a particular line in Vim to make edits—rarely happens.
It's not pretty, one could argue that the mouse is more efficient for such movements, but for me it's faster in practice than moving my hand to the mouse/trackpad, or than looking up a line number (either absolute or relative) and then typing it. And it has no mental burden.