News for hackers. Stuff that matters.

A comment on Ask HN: Best way to do heavy csv processing?.

Re:Ask HN: Best way to do heavy csv processing? by geophile on Sunday October 21, 2018 @11:07AM (#18268338)
This is not a lot of data. Without trying hard, you should be able to import the data within an hour. After that, it's basic postgres optimization. But it's hard to offer advice with such a vague description of the update you are attempting.
[ Parent | on HN ]
Re:Ask HN: Best way to do heavy csv processing? by laxentasken on Sunday October 21, 2018 @01:35PM (#18269065)
Yeah, sorry about that. Well the file will only contain changes that need to be reflected in the old file (which servers as a master). Usually the file contains around 120 fields which then need to be updated. Besides that I will need to update around 150 other fields that serves as descriptions of the original fields.

After that there is some appends of data to be done which usually is a couple of million rows.

[ Parent | on HN ]