This is a pretty small search space in comparison to the size of gpt-3. I would fully expect that the arithmetic table of 0-100 with numbers between 0-100 is present in sufficient density on the internet for this to be memorized by gpt-3.