Re:Command-line Tools can be 235x Faster than your Hadoop Cl...
by usrbinbash on Tuesday March 08, 2022 @04:33AM
(#30598162)
> What about if you had 500 petabytes of data to analyze?
Then we probably need the big tools.
> Big data does exist in the wild.
So does little data.
The problem is that a "one-size-fits-all" approach has become common, not just in data analysis; think of all the low-medium traffic webpages that use giant, complex frameworks and huge distributed systems just to display essentially a small CRUD app that would have been ALOT easier to cobble together in plain JS on a simple LAMP server.
What the article shows is the importance on deciding for the right tool for the job: When I want to plant a little tree in my backyard, bringing one of these
https://upload.wikimedia.org/wikipedia/commons/0/01/Bucket_w...
to dig the hole is proooobably overengineering it a tiny little bit, and will likely take longer than getting a shovel.
Then we probably need the big tools.
> Big data does exist in the wild.
So does little data.
The problem is that a "one-size-fits-all" approach has become common, not just in data analysis; think of all the low-medium traffic webpages that use giant, complex frameworks and huge distributed systems just to display essentially a small CRUD app that would have been ALOT easier to cobble together in plain JS on a simple LAMP server.
What the article shows is the importance on deciding for the right tool for the job: When I want to plant a little tree in my backyard, bringing one of these https://upload.wikimedia.org/wikipedia/commons/0/01/Bucket_w... to dig the hole is proooobably overengineering it a tiny little bit, and will likely take longer than getting a shovel.