If each unique symbol for a number was made up of smaller symbols arranged using a simple algorithm you wouldn't necessarily need to memorize them all. Kind of like Roman numerals, but much a bit more complex.

I don't have to remember the exact symbol for 2,020 in Roman numerals, because I can figure it out. In fact, I have no idea what it would be without sitting down and doing so.

I'm not saying it would be a good system, but it could be usable by mere mortals. I guess you could debate if this would really be "10,000 unique symbols" or just something akin to our current writing system for numbers.