I got the impression that these philosophers were missing how the construction of knowledge (should ?) involve repetition, so one-offs coincidences should be given low probability ?
(Ditto about "cheating" using tricksters, and even worse if they are godlike.)
I think coincidence is be necessary to avoid the scenarios being "justified false beliefs".
I think that merely assigning low probability gives you increasingly better standards for "justified", and then you need some kind of threshold to say if you're sufficiently justified.
The example I was given was "I know there's a sheep in that field, I can see it" but you actually saw a shaggy dog, the sheep which was in the field was hiding behind a bush. Very much a coincidence scenario (like I said, I think they have to be), but also I don't think that fits your model for construction of knowledge involving repetition — instead, I would say that the statement should be "I see what looks like a sheep, therefore I have a justified belief there is a sheep in the field", but normal people wouldn't phrase it like that, and would find it marginally less weird to phrase it as "I see what looks like a sheep," (true) "therefore I know there is a sheep in the field" (false, but the point is they would say "I know" not "I've got a justified belief").
PS: when philosophers are talking about intelligence it's very like neanderthals talking about electricity.
Any feedback loop has some dependency on its own prior state, but then the question is "what's awareness?", and "what's self?".
Awareness is "knowledge or perception" says a random dictionary, and I think we can reasonably state that AI have "perception"… as do single-celled bacteria.
What then is "knowledge"? My Philosophy A-level got as far as pointing out the problems with the definition "justified true belief" and then didn't give an adequate replacement. Me? I think "knowledge" is just the word for our own beliefs. It's like one of the Yes Minister quotes:
"""Oh, that's another of those irregular verbs, isn't it? I give confidential press briefings; you leak; he's been charged under Section 2a of the Official Secrets Act.""" - https://www.imdb.com/title/tt0751832/characters/nm0288621
I know, you believe, the AI is parroting its training set.
The concept of self is a whole extra can of similarly wormy worms.