What matters a lot is genre conventions. This particular singer mostly plays rock, blues, country and reggae. I know the basic mechanics of those four genres, and can often guess where the song is going, even if I've never heard it. Even if I hit a wrong note, it's usually in the chord that's being played (e.g. the 5), and sounds reasonable.
I do the same thing when I get a setlist and just listen to the songs multiple times in the days previous so that I can hear the songs in my head. If often say, "If I can sing the melody, I can play it."
Per the topic at hand, what's also interesting is that I can think that I don't remember a song, but often even years later, if someone starts playing it, I know immediately where to go. Once I find the key, which rarely takes me more than a note or two, I'm good.
But I've also got friends that play traditional Irish music. I can't sit in with them (I've tried) since the chord changes fall at places I don't expect them, and are often different chords than I'd guess. Having a feel for the style of music makes a ginormous difference.
I get a fair amount of that kind of work too, on bass. Mostly jazz, where the repertoire of "standards" is finite. I also go to fiddle jams, where the fiddlers themselves often have no idea what the harmony should be, or how to express that information.
Of course a trick that a layperson might not notice, is that the notes are nowhere close to random.
But I love the challenge of that kind of work if it's with good players.
I guess all that memorization is like the conditioning and training that an athlete might do to prepare for an event, and then at the event you're mostly responding conditions.
And it went OK! In addition to the mental model that you describe, I've also learned that most tunes are more forgiving of changes than one might guess by all of the chord symbols notated so precisely.
I've revived my fake books for another use: To practice treble clef sight reading, which I neglected when I was younger. Also, I think learning melodies helps a lot in terms of mentally organizing and navigating the standards.
There are players who are a lot better than me, and are still attached to the books.
And then you're suddenly hit with all this complexity
The gigs where I get no set list are interesting- after playing for a long time, I just know a lot of songs, or have heard a popular song enough that I know the form and the changes already, and if I can hear the changes in my head, playing them is not a problem. The really interesting bit though is if its a song I've never heard at all. Then, its a game of asking a few questions (what's the key, what's the feel), maybe someone will say it starts on scale degree IV, so you know there's going to be an intro and into the first verse, and you know thats scale degree I probably. Then, you just have to kind of guess your way through the first verse, but you're memorizing the verse at the same time, and usually the second one there's no guessing. And so on. There's also a great deal of watching peoples hands and bodies because there are tells when something is about to happen, and you just get a sense of what they mean.
So there's obviously rote knowledge of music, but there's also this subtext of guessing and watching and listening and mimicry that are a huge part of what is actually going on. The older I get, the more I enjoy thinking about that aspect of making live music.