Solving duplicate data with performant deduplication

Re:Solving duplicate data with performant deduplication by goodroot on Tuesday November 21, 2023 @05:40PM (#38371498)

Hey! Thanks for upvoting.

Happy to answer any questions about deduplication. One thing that's not included in the write-up is that we also address out-of-order indexing alongside deduplication.

[ Parent | on HN ]

Re:Solving duplicate data with performant deduplication by CommanderHux on Tuesday November 21, 2023 @05:58PM (#38371764)

The dataset link seems to be dead. Do you have a mirror?

[ Parent | on HN ]

Re:Solving duplicate data with performant deduplication by goodroot on Tuesday November 21, 2023 @06:15PM (#38371947)

Edit: Updated!

https://mega.nz/folder/A1BjnSYQ#NQe5qhYLVBqiRwhWRmcVtg

Article is updating too.

[ Parent | on HN ]

Re:Solving duplicate data with performant deduplication by CommanderHux on Wednesday November 22, 2023 @11:53AM (#38381685)

Thanks. Is Dedup supported on SQL COPY too?

[ Parent | on HN ]

Re:Solving duplicate data with performant deduplication by nhourcard on Wednesday November 22, 2023 @12:55PM (#38382548)

Not for CSV import via SQL COPY sadly

[ Parent | on HN ]