News for hackers. Stuff that matters.
Re:DuckDB – in-process SQL OLAP database management system by benjaminwootton on Friday February 10, 2023 @04:23PM (#34746054)
SQL is easier and more natural to work with than Pandas.
Re:DuckDB – in-process SQL OLAP database management system by nojito on Friday February 10, 2023 @11:34PM (#34749620)
How would you do df.T in sql?
Re:DuckDB – in-process SQL OLAP database management system by wenc on Saturday February 11, 2023 @03:33AM (#34750631)
df.T is a special Pandas dataframe transpose on the dataframe index and the columns.

DuckDB produces Pandas dataframes, so you would just do df.T. No need to choose between one the other.

But to answer your original question, the SQL analogue to a transpose are PIVOT/UNPIVOT operations which are mathematically rotation operations on invariants (your dimensions). This makes them much more general than a transpose -- which are just rotation operations on the rows/cols. PIVOT/UNPIVOT work on non-square data and allow you to specify different types of aggregations. PIVOT/UNPIVOT keywords are not yet implemented in DuckDB but are on the roadmap if I'm not mistaken.

Re:DuckDB – in-process SQL OLAP database management system by nojito on Sunday February 12, 2023 @07:23PM (#34768676)
>DuckDB produces Pandas dataframes, so you would just do df.T. No need to choose between one the other.

Transforming into a pandas df isn't zero copy.

UNPIVOT and PIVOT are quite verbose compared to df.T.

Re:DuckDB – in-process SQL OLAP database management system by brahbrah on Monday February 13, 2023 @09:35AM (#34773853)
Sometimes yes and sometimes no. For example if you had price data and you wanted to bump prices up a dollar in 2020 how would you do that in sql?

In pandas it’s:

    prices_df.loc['2020'] += 1
If you had a temperature forecast and you wanted to add the expected temperature miss to them, how would you do that on sql?

In pandas it’s:

    temps_df + expected_miss_df