News for hackers. Stuff that matters.
Re:Loading Data into Pandas: Tips and Tricks You May or May ... by wenc on Thursday August 04, 2022 @10:53PM (#32351000)
I’ve mostly replaced pd.read_csv and pd.read_parquet with duckdb.query(“select * from ‘x.csv’) or duckdb.query(“select * from ‘y/*.parquet’).

It’s much faster because DuckDB is vectorized. The result is a Pandas dataframe.

Querying the Pandas dataframe from DuckDB is faster than querying it with Pandas itself.

There are no comments on this comment yet.