I guess you have never read anything about the brain works. The brain is not an organ to memorize, but an organ to forget. It's a filter for information. You won't remember in 30 minutes a series of 10 numbers you are asked to remember by heart, unless you are exposed to it many, many times. Memory works by repetition. That should be a good observation enough to say the brain is not very good at remembering things, since you need to feed it again and again the same information for it to remember it for a while.

Daniel Tammet is not an ordinary person. You are projecting HIS extraordinary capabilities onto everyone else, and you can see that 99.999999 % of the population is NOT anything like Tammet. So your point does not stand.