This is the topic of the language models as tool makers paper https://arxiv.org/pdf/2305.17126.pdf