News for hackers. Stuff that matters.
Re:ClickHouse as an alternative to Elasticsearch for log sto... by phillc73 on Tuesday March 02, 2021 @03:02PM (#26319821)
Yeah, I agree sqldf is quite slow. Fair point.

As you've seen, duckdb registers an "R data frame as a virtual table." I'm not sure what they mean by "yet" either.

Of course it is possible to write an R dataframe to an on-disk duckdb table, if that's what you want to do.

There are some simple benchmarks on the bottom of the duckdf README[1]. Essentially I found for basic SQL SELECT queries, dplyr is quicker, but for much more complex queries, the duckdf/duckdb combination performs better.

If you really want speed of course, just use data.table.

[1] https://github.com/phillc73/duckdf

Re:ClickHouse as an alternative to Elasticsearch for log sto... by kristjansson on Tuesday March 02, 2021 @03:25PM (#26320150)
Didn't realize duckdf was your package. Cool! Definitely something I'll consider for future work, though I spend more time on the Spark-and-Python side of the fence these days.

If you wanted to add corresponding memory benchmarks the value-prop of duckdf might be clearer to those of us that have been scarred by sqldf :).

Re:ClickHouse as an alternative to Elasticsearch for log sto... by phillc73 on Tuesday March 02, 2021 @04:10PM (#26320800)
Sounds like it could be an interesting comparison. I'll look into it.