What's the threshold for when you need to use something like pola.rs instead of just fitting the majority of your data set into memory? If your computer is 8GB and you have ~4-6GB of memory free, you need to be working with a data set that is at least 4GB+, right? Is that a good way to look at it?

Kind of like "do I really need k8s?", "does my workload dictate I need SQL OLAP (Online Analytical Processing) / data frame lazy loading data library?"

what's the general rule of thumb to know "you're missing out by not using existing library like pandas/polars" for somebody who is out of the loop on this kind of stuff