It also sounds like you were overworked and when you started to use LLMs you've stripped yourself of the chance to work with a colleague.
I pictured the normal work from home slack experience.
But I suppose your picture and mine might not have been so different.
If I asked you for your particular experience on something and got an obvious LLM reply, I might say nothing or I might ask if it was an LLM, but either way I’m unlikely to ask you something or trust you ever again. Which also works for you, I guess, since it’d be one fewer person taking up your time. But if you had instead told me “I’m too swamped to help right now” I would’ve instead offered to help take some burden off your back.
If a couple of hundred people in your organisation contact you regularly to ask about procedures, you have a serious documentation problem. If it exists, it will be subpar and/or insufficient. Better someone realises that before you leave and everyone is left hanging. Or perhaps that is part of the goal?
Edit: And now I forgot the most important. When the knowledge the llm retrieved is insufficient to answer colleagues question or the agent skill can not execute the requested task from my colleague, it asks me just for the missing info or skill and with me (the human) in the loop work is done x times faster. Eventually it will replace me and all my colleagues one day. Looking forward to do other stuff then