News for hackers. Stuff that matters.
Re:Ask HN: Identifying duplicate data from a large dataset? by gerenuk on Monday March 18, 2019 @04:29PM (#19424917)
What approach would you be going for initially for deduplicating same urls?
Re:Ask HN: Identifying duplicate data from a large dataset? by verdverm on Tuesday March 19, 2019 @04:15AM (#19428967)
You might look at a real data processing system, like something from the Apache projects