That's a limitation of general purpose computers, not silicon itself. People do build ASICs with memory stored near where it is needed.