In general, for official bots yes - Administrators are the only ones with permissions to add bots to servers, but the permission prompt is usually one that asks for full Administrator permissions and they often don't bother to go back in and remove access to certain channels for eg. music bots.

Another problem is User bots - ie. clients that look like a regular user but are still doing bot stuff or simply ingesting all messages they see, which is easy since Discord's client implementation between bots and users are almost exactly the same. They could have used some wasm/javascript virtual machine thing that would make it hard to figure out what the Discord client is doing and would make it harder to sign in to Discord via a user bot, but then we'd probably have another set of HN users complain about Discord hiding stuff from users or something.