News for hackers. Stuff that matters.
Re:The Rise of SQL-Based Data Modeling by swalsh on Thursday January 30, 2020 @08:49AM (#22191236)
" Up till a few years ago, the traditional way of managing data (in SQL-based RDBMSs) was considered a relic of the past, as these systems couldn't scale to cope with such a huge amount of data. "

I must have missed the boat on this one. I remember in 2010 there was a brief period of time where NoSQL was in fashion, but it rightfully died pretty quickly to a small set of specialized use cases. There have been some cases where big data systems have replaced more traditional rdbms systems, but now you can use SQL for those too (like Hive SQL).

SQL is the one skill that has not become obsolete in the course of my career. Frankly I've started relying on it more, because it never goes obsolete. Also it's fast as hell. When I first started my career I did C#, and the .net Framework 2 was fairly new. Since then WinForms, and WebForms have gone away. ORM's changed, Javascript changed. Then I moved to Ruby, and Python, and PHP. Those ecosystems have evolved too. But the one thing that I learned 15 years ago, that I still use every day is SQL.

Re:The Rise of SQL-Based Data Modeling by gwbas1c on Thursday January 30, 2020 @09:12AM (#22191369)
> I must have missed the boat on this one. I remember in 2010 there was a brief period of time where NoSQL was in fashion, but it rightfully died pretty quickly to a small set of specialized use cases. There have been some cases where big data systems have replaced more traditional rdbms systems, but now you can use SQL for those too (like Hive SQL).

A few weeks ago I had a lead server-side architect call me up and ask if my client could tolerate eventual consistency. My answer was that "he could get away with that 10 years ago but that NoSQL went out of fashion and SQL came back in again."

To put it in a different context: Some people just go through periods in their life where they don't keep up and don't realize what was "new and exciting" 10 years ago turned out to be smoke and mirrors.

Re:The Rise of SQL-Based Data Modeling by pdexter on Thursday January 30, 2020 @10:49AM (#22192188)
What's the connection between NoSQL vs SQL and eventual consistency?
Re:The Rise of SQL-Based Data Modeling by dragonwriter on Thursday January 30, 2020 @11:13AM (#22192459)
NoSQL is a bad name that really doesn't have anything to do with SQL per se, but is a shorthand for non-relational datastores (no one calls non-SQL relational systems “NoSQL”), which, while they depart from the relational model in a number of different ways, typically include eventual rather than strong consistency to support liveness in a distributed context.
Re:The Rise of SQL-Based Data Modeling by cle on Thursday January 30, 2020 @11:07AM (#22192389)
NoSQL in some cases trades off consistency for scalability (generally by sharding data, so propagating data across shards introduces eventual consistency).

As you can imagine, data that reside in a single shard don’t need to be propagated, so you can get strong consistency easily (e.g. fetching a single document). It’s when you start doing queries, joins, aggregations, etc. that you start encountering eventual consistency.

Re:The Rise of SQL-Based Data Modeling by jsight on Thursday January 30, 2020 @11:49AM (#22192821)
Mainly that most people who ask that question are using it as a lead-in to promoting a NoSQL product.
Re:The Rise of SQL-Based Data Modeling by cle on Thursday January 30, 2020 @11:12AM (#22192447)
NoSQL is very much alive and kicking, it’s just past the peak in the hype curve. There is a FAANG in particular who is notorious for banning relational databases outright because of their maintenance and scaling problems for their use cases.

As others have said, if you operate at a certain scale and can get away without complex joins and aggregations (which you should avoid anyway if you’re operating a high availability service), then NoSQL is a great option. It’s easier to scale out (often you just turn a knob) and its behavior is simpler and easier to predict.

Re:The Rise of SQL-Based Data Modeling by gwbas1c on Thursday January 30, 2020 @01:13PM (#22193674)
The phrase that comes out when someone proposes NoSQL is "we're not Google / Facebook."

Most applications don't need to run at "Google scale," thus they don't need the expense of relaxing ACID.

Re:The Rise of SQL-Based Data Modeling by cle on Thursday January 30, 2020 @10:55PM (#22198888)
Of course, consider your use cases. There are good reasons to use NoSQL, and excellent reasons to use RDBMS. I use both professionally and appreciate the benefits of each.
Re:The Rise of SQL-Based Data Modeling by oarabbus_ on Thursday January 30, 2020 @12:25PM (#22193229)
What FAANG _bans_ relational DBs?
Re:The Rise of SQL-Based Data Modeling by jimbokun on Thursday January 30, 2020 @10:21AM (#22191901)
The nugget of truth was that joins and transactions are very difficult to implement in a performant manner in large scale, distributed systems. So it was possible to write very highly scalable databases if you give up on "SQL", which was a stand in for joins and transactions.

Since then, developers have been coming up with clever ways to implement distributed joins and transactions more efficiently and scalably on top of distributed database engines, and thus "the return of SQL".

Re:The Rise of SQL-Based Data Modeling by 6gvONxR4sf7o on Thursday January 30, 2020 @11:12AM (#22192451)
Where can I learn more about these clever developments?
Re:The Rise of SQL-Based Data Modeling by jimbokun on Thursday January 30, 2020 @11:54AM (#22192870)
I find CockroachDB one of the most interesting systems adding distributed transactions, joins and SQL support on top of a highly scalable key value store engine:

https://www.cockroachlabs.com/docs/stable/architecture/overv...

Re:The Rise of SQL-Based Data Modeling by jwatte on Thursday January 30, 2020 @12:48PM (#22193440)
Re:The Rise of SQL-Based Data Modeling by collyw on Thursday January 30, 2020 @10:27AM (#22191977)
The youngsters in my work don't really know SQL.

I am probably see as one of these old developers that refuses to get with the times (rather than seeing NoSQL for a step backwards that it is). Its this constant churn of bullshit that makes me want to do a different job after nearly 20 years (as well as the nonsense that is the hiring process).

Re:The Rise of SQL-Based Data Modeling by mumblemumble on Thursday January 30, 2020 @10:51AM (#22192217)
This, I think, is the true cost of technologies like ORM. And it's hard to notice, because it doesn't show up right away; it's a generational thing.

It reminds me a bit of a paper I saw recently on The Morning Paper called Ironies of Automation: https://blog.acolyer.org/2020/01/08/ironies-of-automation/

Re:The Rise of SQL-Based Data Modeling by larrydag on Thursday January 30, 2020 @09:02AM (#22191318)
Unfortunately for this data analyst NoSQL is alive and well. It is being rolled out to enterprise levels without the downstream impacts fully realized. This requires all sorts of ad-hoc ETL data methods which in turn puts more time and strain on budgets and resources. At least this is true in my organization.
Re:The Rise of SQL-Based Data Modeling by baldfat on Thursday January 30, 2020 @10:19AM (#22191888)
I think there is much more non-SQL then the OP realizes. I have been protesting non-SQL, SQL is more powerful than most realize. Also the term "Big Data" is being used 99% of the time about data bases I could load into my ram but counted in millions. This then causes people to search for things and find non-SQL.
Re:The Rise of SQL-Based Data Modeling by EdwardDiego on Thursday January 30, 2020 @11:57PM (#22199195)
That said, every big data tool I've encountered in current usage supports SQL - very few people writing Map/Reduce jobs these days. SQL has become the lingua franca of data analysis.
Re:The Rise of SQL-Based Data Modeling by exabrial on Thursday January 30, 2020 @09:31AM (#22191500)
> brief period of time where NoSQL was in fashion, but it rightfully died pretty quickly

That what is not standardized, dies

Re:The Rise of SQL-Based Data Modeling by oyoun on Thursday January 30, 2020 @08:58AM (#22191294)
I totally agree with you. And SQL keep evolving
Re:The Rise of SQL-Based Data Modeling by pastor_elm on Thursday January 30, 2020 @10:28AM (#22191984)
So long as fresh faced developers arrive on the scene, there will always be people looking to get around using SQL.
Re:The Rise of SQL-Based Data Modeling by oyoun on Thursday January 30, 2020 @09:13AM (#22191375)
And database could scale up at this time: Pan-STARRS is one of the bigger database with PB
Re:The Rise of SQL-Based Data Modeling by jstrong on Thursday January 30, 2020 @09:39AM (#22191544)
> Also it's fast as hell

compared to what? excel?