News for hackers. Stuff that matters.

A comment on Command-line Tools can be 235x Faster than your Hadoop Cluster (2014). Parent comment

Re:Command-line Tools can be 235x Faster than your Hadoop Cl... by markus_zhang on Tuesday March 08, 2022 @08:25AM (#30599599)
What I really dislike modern cloud DWH such as Snowflake is that it hides a lot of things from me. Since I'm not a CTO who worries about not delivering, but a junior DE who actually wants to learn things, I really prefer that things were done in the old ways where we had to manage our own infrastructure and our own code for ETL. These kinds of things can not be learned "just for fun" because one has to work in a real environment.
[ Parent | on HN ]
Re:Command-line Tools can be 235x Faster than your Hadoop Cl... by manigandham on Tuesday March 08, 2022 @09:00AM (#30599928)
What do you mean? Of course you can still learn them "just for fun" if you want. There are plenty of columnar data warehouses (memsql, greenplum, vertica, clickhouse, etc) and data processing frameworks (spark, flink, etc) that you can look at, implement and run yourself.

It's all using the same principles underneath.

[ Parent | on HN ]
Re:Command-line Tools can be 235x Faster than your Hadoop Cl... by markus_zhang on Tuesday March 08, 2022 @09:06AM (#30599993)
What I'm saying is that you can surely scratch the basics from personal use, but it's completely different from real usage and such can only be trained on job. Now that those jobs are fewer as everyone goes on cloud.
[ Parent | on HN ]
Re:Command-line Tools can be 235x Faster than your Hadoop Cl... by tharne on Tuesday March 08, 2022 @07:33PM (#30609465)
> What I really dislike modern cloud DWH such as Snowflake is that it hides a lot of things from me.

This is by design. The less skill required to use a tool, the less some CTO has to pay the person using it.

[ Parent | on HN ]