The answer is probably AI middleware (something to stand between you and the LLM and help tune your prompts). We did this for a hackathon recently and it worked quite well. We also open-sourced OpenPlayground so you can test prompt tuning against multiple LLMs.