Containing an infinite amount of data and giving accurate answers are saying different things though. It can have finite memory and still give 100% accurate answers by batching from disk.

Chunking / iterative processing like this is an approach people commonly use with pandas which otherwise isn't great for data sets too big for RAM.

http://pandas-docs.github.io/pandas-docs-travis/io.html#iter...