Doing a database join with CSV files

Re:Doing a database join with CSV files by haddr on Wednesday January 01, 2020 @07:29AM (#21927781)

In R this is rather straightforward, and you can also achieve it without additional libraries:

  weight <- read.csv("weight.csv")
  person <- read.csv("person.csv")
  merge(weight, person, all=TRUE)

Of course, nowadays you would use data.table, but still the merging logic would be exactly the same.

[ Parent | on HN ]

Re:Doing a database join with CSV files by atombender on Wednesday January 01, 2020 @03:52PM (#21930546)

That loads all the data into memory, though. Fine if you're joining all of it, but with the Postgres example you can throw the whole SQL language at the data, and it will be streamed and pipelined efficiently just like with normal Postgres tables.

[ Parent | on HN ]

Re:Doing a database join with CSV files by nkozyra on Wednesday January 01, 2020 @07:52AM (#21927838)

I never got deep into R as we had python or R as options in school for data projects so forgive me but where is the database join in this scenario?

[ Parent | on HN ]

Re:Doing a database join with CSV files by haddr on Wednesday January 01, 2020 @08:53AM (#21927998)

Original article doesn't say anything about "database join". It's about joining two datasets by some common ID.

R in this case fits the bill and even allows for some relational algebra here (e.g. INNER JOIN would be merge(X, Y, all=FALSE), LEFT OUTER JOIN: merge(X, Y, all.x=TRUE), etc...)

[ Parent | on HN ]

Re:Doing a database join with CSV files by stilisstuk on Wednesday January 01, 2020 @08:53AM (#21927999)

Merge joins the two dataset on identical columns. All=True is an outer join.

[ Parent | on HN ]