Consider reframing. Is the problem really that the task is tedious? Or that it's required? Maybe you should be leaning more on deterministic automations (which the LLM can help set up) rather than LLM-powered ones.
- Research and buy me a domain name for xyz product, set up all of its DNS the way I want for this product, set up email, set up app store submissions including screenshots and description, set up firebase notifications, set up my Google cloud project and any apis I need to enable for this app, deal with app store review feedback.
- Do an audit of my books, make sure every transaction has a receipt uploaded, and make sure the receipt matches the journal entry. Pull statements from all my my bank accounts and reconcile against my books. Fix mistaken transactions with my confirmation.
- Freely push to my source control, but only to specific branch prefixes (surprised that GitHub, gitlab, gitea, etc don't have this built in already). Merge with my confirmation.
Not to mention that I only have to have a single key that gives access to all my services. If the key leaks, I just have to rotate one key because the agent never has access to the 30 upstream service keys. If I need to give weaker or stronger keys for a specific agent, I can easily configure that in one place (and I can even request that the agent make that change for me with my confirmation). I don't need to create 20 new keys every time, and I don't need to remember the intricacies of every upstream service's UI for creating new keys.
The best solution I've been able to come up is using an ephemeral vm (on an isolated network) per task, using --dangerously-skip-permissions so it doesn't bother me. I couple that with an API proxy that lets me restrict the permissions it has to external services. So e.g. where a quickbooks API key gives the holder "god mode" access, the proxy lets the agent do certain read commands without confirmation, but all write endpoints except uploading receipts require confirmation on my phone before they'll be allowed to hit the quickbooks servers. I no longer need to trust that the agent will interpret my readonly wishes correctly.
Tooting my own horn here, but this setup works incredibly well! I've been able to connect dozens of services safely where I absolutely do not trust giving raw, overpowered credentials or oauth integration to the agent directly. I actually just released the API proxy today at https://keycontain.com