On the other hand, Searle confidently states that a perfect quantum mechanical simulation of a human brain would not be conscious. This is because it somehow lacks the intentional states of the brain it's perfectly simulating. He doesn't feel the need to justify that a human mind has these states, because our daily direct experience of consciousness cannot seriously be in question. The daily direct experience of consciousness that a perfect simulation of a human brain would assert that it feels is, somehow, different and we can't trust it.
At every step in his argument there's caveats, misleading allegory, and dismissals that are so much of a stretch it seems pretty obvious he's retroactively arguing from a position that our biological conscious minds are somehow special and mystic.
That solving a problem, or an array of problems that a conscious mind can solve may be perfectly able to be solved by a non-conscious machine. That the consciousness part may not be relevant in the problem-solving process, and that consciousness may be unique to living beings who have distinct physiology, anatomy, ways to process information, that computers as they exist now do not have. That a simulation is not necessarily the same thing as the thing it simulates, and does not have all of its properties.
These are good questions none of which are mystic and which following from Searle's argument. And I think you're really underestimating to what degree people are lured in by the 'magic' of neural nets because else we wouldn't be in this thread. A neural net that does some function approximation is not 'slightly conscious' unless we're going full in on panpsychism. Even from a functionalist perspective, the neural net performs nothing that is reasonably involved in what we consider to be responsible for consciousness. Nobody ever claimed a TI-83 is 'slightly conscious', or that Stockfish is merely because it plays exceptional chess.
Searle, simplified holds that consciousness is a property of very particular biochemical systems, not a disembodied thing that exists independent of the matter it inhibits.
It's the rational-oriented circles ironically enough who almost dabble in a Cartesian Dualism by positing "computation" or "minds" as distinct from matter. In fact with on occasion almost religious-sounding implications.