> When I have large data, then I directly write SQL queries and run them against the database

what database is that? For example PgSQL will be XX-XXX times slower on OLAP queries than duckdb/polars/datafusion from various reasons.