News for hackers. Stuff that matters.

A comment on Querying Parquet with Precision Using DuckDB. Parent comment

Re:Querying Parquet with Precision Using DuckDB by nyokodo on Friday June 25, 2021 @09:51PM (#27638331)
> I tend to work in Python not Scala

Fair enough. I’m more comfortable in Scala than Python. Have you tried using a local Jupyter notebook? That would seem to tick all the boxes. https://jupyter.org/install

[ Parent | on HN ]
Re:Querying Parquet with Precision Using DuckDB by wenc on Friday June 25, 2021 @10:10PM (#27638426)
I do use Jupyter and the boilerplate code above is what I usually deploy in Jupyter.

Spark syntax is still just a touch more complicated than necessary for my needs because it's solving the much more general distributed computing problem. (not to mention the impedance mismatch -- Spark is very much a Scala native app, and it feels that way to a Python user).

DuckDB on the other hand seems to provide a quick way to interact with local Parquet files in Python, which I appreciate.

[ Parent | on HN ]