His point is that while we're over here arguing over whether a particular AI is "really" doing certain things (e.g. knows what it's doing), it can still cause tremendous harm if it optimizes or hallucinates in just the right way.