Re:WARC'in the Crawler
by marginalia_nu on Friday December 22, 2023 @09:51AM
(#38734643)
Hadoop is a bit of a nuisance in this general corner of Java. It's got a propensity for integrating deeply with cluster adjacent technology in a way that is very difficult to root out.
Kind of a pity since it has the effect of making things that could be very easy, such as reading and writing parquet files, much harder than it needs to be in Java.
Kind of a pity since it has the effect of making things that could be very easy, such as reading and writing parquet files, much harder than it needs to be in Java.