One middle point between automated and manual signing is, as usual, key rotation: have the signing keys expire in a short duration of time (say 2 weeks) and manually push them every week, so that the window of attack is as small as possible.
1. A release candidate X is tested in a CI
2. If tests pass, the CI sends a notification to the build server. The notification is "prep for release package <hashid>"
3. Build server pulls code from the repo, matches it against "ready, CI passed" notification and builds the package/packages.
Compromise of the entire CI/dev chain would be contained as the builders act as a new pipeline entry point running in parallel of the CI using pull method. To compromise keys located on a build server one would need to either get access to it via whatever the method of remote access the server has ( which should be nearly none ) or figure out how to compromise the code running on the builders using the input from a repo that passed CI.
You have to define what the signature means.
IMHO it is fine for it to mean "this software was built on our build server from a well-defined state of the source code, which is only changable by our employees and contractors, and for which we have the full change log". So I deploy the code signing key to build servers, which is the only place where it is used.
I'm interested in what alternative meaning you would give to a signature. I have considered the possibility of tying it to the QA processes, but then a build can only be signed after checking it manually, which is problematic when many signatures are needed at multiple packaging layers (exe/dll, msi, setup.exe).