Sure there are better methods for arithmetic but arithmetic is extremely quantifiable with rigid steps. What happens when you step out of that kind of domain ? Like the above blog. or Code documentation. For example, you can paste new documentation in a gpt-4 instance and it will use it for your queries as if it trained on it.
Basically Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589. and you kind of get the feeling of that from the previous paper.
You can't do it for everything, but if you can generate code and run it outside the LLM, you should.
Gpt-3.5 doesn't need the algorithm prompt for every single query. It just needs it for every query that requires arithmetic. Much more feasible.
A more practical thing to do for algorithms is probably to use ChatGPT to help you write the function you need.