I have the exact same problem. Interestingly, it's a problem for me both in my native language and in several other languages I learned to fluency. So clearly, the "hears numbers" part of my brain is deficient.

I've actually practiced to try to overcome this, and made no progress.

On the other hand, I'm quite good at visual calculations, and also at remembering numbers when I see them. So if I bother writing down a phone number, I can easily remember it for a long time.

But hearing it? Nothing.

If you don't care about the setting: 1. Click "Play game" 2. Press 2, Enter, Star once it loads.

Make 24. Enter the two intermediate results. Numpad plus repeats the question.

I don't know if this is a good place to start if you have trouble hearing but might be fun for practice anyway.