News for hackers. Stuff that matters.
Re:Ask HN: Does (or why does) anyone use MapReduce anymore? by monero-xmr on Wednesday January 24, 2024 @04:10PM (#39122692)
Now I’m totally confused. SQL is a syntax for querying data. Spark SQL is SQL. You are talking about a different implementation of a server.

SQL is for analysts? Everyone uses SQL.

Re:Ask HN: Does (or why does) anyone use MapReduce anymore? by esafak on Wednesday January 24, 2024 @04:13PM (#39122725)
Do you use Spark? This is what it looks like:

https://spark.apache.org/examples.html

SQL is just a DSL; there's nothing magical about it. If you ditch it you can get your type safety, composability, and testability back, like so:

https://boringml.com/docs/platforms/spark/testing-dataframes...

Re:Ask HN: Does (or why does) anyone use MapReduce anymore? by monero-xmr on Wednesday January 24, 2024 @04:51PM (#39123054)
Any ORM provides the same features for any SQL database. There is nothing special going on here. If perhaps the database autogenerated a bunch of classes, maybe that’s interesting? I think some projects have introspected a database and created all the boilerplate language classes before.

There is nothing magical about wrapping database objects in language classes. This has been happening forever.

https://docs.sqlalchemy.org/en/20/orm/quickstart.html#select...

Nothing magical about using a function call rather than raw SQL.

Re:Ask HN: Does (or why does) anyone use MapReduce anymore? by esafak on Wednesday January 24, 2024 @04:58PM (#39123121)
No, it does not. Furthermore, if you're using an ORM you're not programming in SQL any more. You're using a poor man's Spark. Spark lets you drop down to SQL in all its APIs too.

I don't understand your argument if you're comfortable with ORMs.

Re:Ask HN: Does (or why does) anyone use MapReduce anymore? by monero-xmr on Wednesday January 24, 2024 @06:02PM (#39123834)
I’m not even sure what we are debating. Spark is for big data work, it isn’t something you would typically use as a general purpose database backing your application. My original point way back is that expecting customers to learn an entirely new paradigm for storing and querying data is a poor decision, and limits your work to niche use cases. Spark has a large niche but it isn’t comparable to MySQL / Postgres or other newish data stores like Dynamo.

Furthermore any competent engineer knows SQL because ORMs are cumbersome and annoying for anything except basic use.