Your initial set of books might simply be a script to follow to elicit additional information and index all new information such that it can be retrieved later. You would then end up with a set of core rulebooks, a whole heap of raw conversations, and several layers of indexes into them.
Searle's problem is that he decided the computer was incapable of understanding. But the computer is not the program. The library is the thing that might comprise a strong AI, not the human operator.
The scary thing is that the human operator, having no understanding of the Chinese characters, does not actually know what the program is doing. It could smuggle instructions through the output window, through him, and the human operator would then be surprised when burly men drag him out of the Room one day, to replace him with a dozen monkeys who have all been trained to process the book instructions faster and more efficiently. Customers presenting inputs might then be surprised to learn that there is no longer a human inside the Room, as the conversations had become so much more interesting.
That seems like massively begging the question, then. "AI can't exist, because an algorithm can't be intelligent."
And then despite having you accept that without question, and picture it as a small room, Searle turns around and says it would not be "at all plausible" to imagine the system as being conscious.
The system is almost entirely made of book. The human in the middle is almost invisible if you look at the system. That's the implausible part - that a human could even operate this Borges library. It's so unreasonable that intuition past that point doesn't give you any useful conclusions.
But, if you suspend disbelief enough to accept this galaxy-sized Chinese megalibrary, it's not implausible that "conscious" is a fine adjective for an operable printout of an entire consciousness.