Breaking an RSA public key consists of two phases. The first phase is massively parallel and indeed doesn't require much storage.

However, the second phase requires solving a massive set of linear equations and requires a more conventional big computer.