Re:Show HN: How to analyse 100 GB of data on your laptop wit...
by kthejoker2 on Monday December 02, 2019 @06:27PM
(#21688250)
Think you'd want to compare with a columnar database since this is an analytic workload, not a transactional one.
And a 4 node Spark cluster with Parquet or Arrow files and a Scala job will compare favorably to this because it's also a lazy evaluator and the benchmark problems here are embarrassingly parallel.
And a 4 node Spark cluster with Parquet or Arrow files and a Scala job will compare favorably to this because it's also a lazy evaluator and the benchmark problems here are embarrassingly parallel.
PS this is a very cool project!