So you already protected most of it - no write access and scoped keys fix most of the issues.

Other potential problems:

- bad checkout location may mean unexpected content is available via .git paths on the web

- anyone with access to the server can copy the key and have external access to both the history of the project and all new commits - they can see the PRs with proposed security fixes before they get merged

- repository may contain domain names, credentials and other things which don't need to be deployed, but can be useful for the attacker doing recon

- potentially exposing information about customers if they got mentioned in the history

It's not terrible to use git directly. There are just ways you can deploy a little bit better if it's worth your time investment.