Perhaps tools trained into the model rather than exposed through prompting would mitigate the performance hit (but might affect model quality?).