News for hackers. Stuff that matters.
Re:MongoDB shares jump more than 30% in $192M IPO by growse on Thursday October 19, 2017 @01:42PM (#15509914)
> That said, it's pretty hard to make sense of when you would want non-relational dbms these days. Write-scaling is still a pretty obvious reason, though things like Citus might help here.

Personally, I find a non-relational database useful when my data model is non-relational.

Re:MongoDB shares jump more than 30% in $192M IPO by hungerstrike on Thursday October 19, 2017 @01:53PM (#15510011)
What is an example of non-relational data?

A tree is relational. Each child has a relation to its parent.

I can't imagine data that has no relation (no connection) to anything else. Maybe what you meant was heterogeneous (e.g. data elements that do not all have the same attributes) - but even then I can't readily come up with an example.

Re:MongoDB shares jump more than 30% in $192M IPO by balfirevic on Thursday October 19, 2017 @02:23PM (#15510314)
Word "relational" in relational databases does not stand for relationships between the tables [0]. That being said, relational model is a great fit for plenty (most?) of the data.

[0] - https://en.wikipedia.org/wiki/Relation_(database)

Re:MongoDB shares jump more than 30% in $192M IPO by hungerstrike on Thursday October 19, 2017 @04:30PM (#15511298)
Thanks for the technical correction. That also being said, I think that definition is terrible and that anybody trying to understand what a relational database is would be better off not reading that article.

Yes, I agree that most types of data can be stored in tabular form (or an "n-ary relation" per Wikipedia). I'm just wondering what concrete types of data one would rather store in a document.

I don't think there is a good example. The decision to store some data outside of an RDBMS must have more to do with the processing model or something else.

Re:MongoDB shares jump more than 30% in $192M IPO by growse on Friday October 20, 2017 @05:56AM (#15514619)
> I don't think there is a good example. The decision to store some data outside of an RDBMS must have more to do with the processing model or something else.

What else other than the processing model and business requirements would determine how you model and store your data?

Re:MongoDB shares jump more than 30% in $192M IPO by hungerstrike on Friday October 20, 2017 @12:08PM (#15516897)
That was my point. The person I responded to said that they only store "non-relational data" in NoSQL so I was asking what that was.
Re:MongoDB shares jump more than 30% in $192M IPO by balfirevic on Saturday October 21, 2017 @06:49AM (#15521420)
To clear up the possible confusion, the person you are currently responding to said that, not me.
Re:MongoDB shares jump more than 30% in $192M IPO by dandermotj on Thursday October 19, 2017 @02:39PM (#15510446)
Sparse heterogeneous data is often the type of data stored in NoSQL dbs. Modelled in a relational way, this produces many tables with many NULL fields, while keeping it in a key value format is neat and tidy.

I'd recommend the paper What Goes Around Comes Around[1], the first paper in Readings in Database Systems[2]

[1] https://scholar.google.com/scholar?cluster=73661829057771494... [2]redbook.io

Re:MongoDB shares jump more than 30% in $192M IPO by wvenable on Thursday October 19, 2017 @02:46PM (#15510518)
> Sparse heterogeneous data is often the type of data stored in NoSQL dbs.

I still can't imagine what sparse heterogeneous data exists in the world that makes sense to store. Any type of querying or processing requires some kind of structure (even if implicit in the code) which you can just put in different table structures.

You have to make sense of data to process it and that kind of implies a structure, doesn't it? Am I missing some obvious example of heterogeneous data?

Re:MongoDB shares jump more than 30% in $192M IPO by threeseed on Thursday October 19, 2017 @04:16PM (#15511196)
Customer Analytical Record / Feature Engineering Store

One customer column, tens of thousands of attribute columns.

If you need everything about a customer it is a single, O(1) fetch operation which makes it perfect for driving chat bots, call centres, websites, operational decisioning engines, dashboards etc. Almost every large company will have one of these.

You can't really do it in relational systems properly because (a) you hit the column limit, (b) often it is sparse i.e. lots of NULLs everywhere, (c) you need this system to be distributed since it often gets a lot of load.

Re:MongoDB shares jump more than 30% in $192M IPO by rwallace on Thursday October 19, 2017 @05:49PM (#15511829)
What would the attribute columns consist of? My experience has been with named columns defined individually by humans, of which I've never seen more than a few hundred; how do you get tens of thousands? Are they a different kind of thing?
Re:MongoDB shares jump more than 30% in $192M IPO by threeseed on Thursday October 19, 2017 @06:37PM (#15512124)
Most companies who do it purely by humans can easily get into the thousands of attributes. Have seen it many times before where you hit the column limit of a SQL database.

But where you get into tens/hundreds of thousands is when you have machine learning models automatically selecting and storing important features from the data.

Re:MongoDB shares jump more than 30% in $192M IPO by growse on Thursday October 19, 2017 @04:32PM (#15511321)
Tick (market) data is another good example of this. A given 'Tick' is just an event that can have any of up to thousands of different attributes set (often just a handful).
Re:MongoDB shares jump more than 30% in $192M IPO by bpicolo on Thursday October 19, 2017 @02:42PM (#15510481)
Re:MongoDB shares jump more than 30% in $192M IPO by threeseed on Thursday October 19, 2017 @04:12PM (#15511169)
No.

EAVT is great as an intermediate format but it is absolutely useless to query for since most of the time you are trying to find a set of attributes for a given entity i.e. full table scan.

What you want is a "wide table". One entity column and all the attribute columns to the right. Often with most of the values set to null.

This is the dream use case for MongoDB since it you can ignore sparse values yet when you query it via their drivers it will appear as a wide table. You can't do this at all in PostgreSQL since you will hit a column limit.

Re:MongoDB shares jump more than 30% in $192M IPO by flukus on Thursday October 19, 2017 @06:43PM (#15512160)
> EAVT is great as an intermediate format but it is absolutely useless to query for since most of the time you are trying to find a set of attributes for a given entity i.e. full table scan.

This is what indexes are for. An index on the entity id should avoid any full table scans.

Re:MongoDB shares jump more than 30% in $192M IPO by threeseed on Thursday October 19, 2017 @11:30PM (#15513365)
EAVT table with 100 million entities and 10000 attributes = one trillion row table.

And you want to build indexes on half the table ?

Good luck with that.

Re:MongoDB shares jump more than 30% in $192M IPO by flukus on Thursday October 19, 2017 @11:51PM (#15513443)
> Often with most of the values set to null.

Your math is at odds with your own requirements, null values don't need a row.

Re:MongoDB shares jump more than 30% in $192M IPO by threeseed on Friday October 20, 2017 @01:02AM (#15513635)
You clearly don't understand what you're talking about.

Sparcity is an issue for the wide table not the EAVT form.

Re:MongoDB shares jump more than 30% in $192M IPO by kuschku on Thursday October 19, 2017 @05:23PM (#15511664)
> You can't do this at all in PostgreSQL since you will hit a column limit.

JSONB is designed for exactly this, isn’t it?

Re:MongoDB shares jump more than 30% in $192M IPO by threeseed on Thursday October 19, 2017 @06:39PM (#15512139)
Sure. But MongoDB is far better at scaling, has infinitely better drivers (including Spark) and is about an order of magnitude faster than PostgreSQL for partial updates.

The lack of a Spark driver alone renders PostgreSQL useless for most companies.

Re:MongoDB shares jump more than 30% in $192M IPO by growse on Thursday October 19, 2017 @04:35PM (#15511344)
Any data can be modelled in a relational way, I guess. It doesn't mean it's the best representation.

Try modelling a cyclic graph in a relational way and you'll quickly tie yourself in knots trying to update and query it.

The point is, relational databases a great for storing data that you've decided to model relationally. If you decide not to, then you probably want some other sort of database.

Re:MongoDB shares jump more than 30% in $192M IPO by gaius on Friday October 20, 2017 @02:29AM (#15513868)
The point is, relational databases a great for storing data that you've decided to model relationally. If you decide not to, then you probably want some other sort of database.

Sure. But that database is not MongoDB.

Re:MongoDB shares jump more than 30% in $192M IPO by growse on Friday October 20, 2017 @05:52AM (#15514611)
Oh, I totally agree!

The "you only need relational databases" mantra bugs me though, because it's so obviously not true.

Re:MongoDB shares jump more than 30% in $192M IPO by mygo on Friday October 20, 2017 @11:11AM (#15516391)
such a naive outlook. MongoDB has two things that are being incredibly overlooked right now: 1- company stability. So you don't have to redo your database in 3 years. 2- developer community / pervasiveness. So you have an easier time integrating it into your projects, even with its technical shortcomings.

I bet two years ago there was someone out there saying "if you're going to do NoSQL you better use RethinkDB over MongoDB"

How great the technology is, is absolutely not the only factor. Good thing people can take in many different factors when making their decisions.

Re:MongoDB shares jump more than 30% in $192M IPO by pcsanwald on Thursday October 19, 2017 @03:04PM (#15510657)
instead of non-relational, think of it as denormalized. I also can't think of any cases where you wouldn't want some relationships. I can absolutely rattle off tons of cases where applications benefit greatly from some kind of denormalization.

In years past, people called these "data warehouses" and essentially took snapshots of their production DBs and denormalized the hell out of them so that aggregations wouldn't crash the server.

Re:MongoDB shares jump more than 30% in $192M IPO by bpicolo on Thursday October 19, 2017 @01:47PM (#15509953)
Sure. The most direct argument against that is that Postgresql jsonb is just non-relational data support in a first class relational DB, which is pretty great, so to an extend you get the best of both worlds, though I'm sure you can find a case where it's not quite optimal vs some nosql db.

This talk is a pretty nifty perf overview:

https://www.percona.com/live/e17/sessions/high-performance-j...

That said, if you know beforehand that horizontal scaling will be a crucial factor, probably postgres isn't the first choice. But with how fast CPUs are these days it's usually not important for a long time.

Re:MongoDB shares jump more than 30% in $192M IPO by jsjohnst on Thursday October 19, 2017 @02:11PM (#15510190)
> But with how fast CPUs are these days it's usually not important for a long time.

That’s a very naive statement to make.

Re:MongoDB shares jump more than 30% in $192M IPO by takeda on Friday October 20, 2017 @12:35PM (#15517100)
If you grow and you hit hardware limits then congratulations you are the next facebook/google/etc. Also there's nothing stopping you from switching your current database to something else.

Absolutely majority of the companies will do just fine, because the hardware improves faster than their demands. Starting out with a distributed system "because one day we might need it" is just silly, because chances are you'll never hit it, and you'll have to pay for the overhead of having a distributed system (which is non trivial).

Actually my company started using PG and had presentation and someone asked if we considered a distributed database so we can scale. The presenter nicely said it was evaluated and this solution worked best, but that was too nice.

1. It's only about 100GB of data

2. The hardware is barely utilized, we didn't tune it (except some standard memory settings), because there's no need yet.

3. Our data is relational (in fact most data from most companies is relational)

Re:MongoDB shares jump more than 30% in $192M IPO by bpicolo on Thursday October 19, 2017 @02:21PM (#15510284)
Is it? Because there are a ton of companies out there running postgres, mysql and scaling workloads just fine.
Re:MongoDB shares jump more than 30% in $192M IPO by jsjohnst on Thursday October 19, 2017 @02:41PM (#15510466)
I upvoted you because you’re technically right for many cases, but if you’d ever dealt with a database you couldn’t scale further (we were using multiple of the largest instances EC2 had at the time) and new data was flowing in faster than you could delete it and downtime wasn’t an option, you too would hate any predecessors who said “you can always scale higher” before things got so bad it was almost impossible to recover.
Re:MongoDB shares jump more than 30% in $192M IPO by bpicolo on Thursday October 19, 2017 @02:53PM (#15510581)
Yeah. There are definitely cases where the scaling options make sense along different axis for sure. My point is mostly that relational DBs tend to strike a good default ground if you aren't sure what axis you'll care about.
Re:MongoDB shares jump more than 30% in $192M IPO by threeseed on Thursday October 19, 2017 @04:08PM (#15511135)
PostgreSQL JSONB doesn't have a dedicated driver though i.e. you can't do partial updates via the JDBC/ODBC drivers.

Which means you can't use it for any big data/analytics use cases. MongoDB has fantastic client libraries e.g. Spark, Java.

Re:MongoDB shares jump more than 30% in $192M IPO by takeda on Thursday October 19, 2017 @04:48PM (#15511417)
Why wouldn't you be able to do partial updates with JDBC drivers?
Re:MongoDB shares jump more than 30% in $192M IPO by kuschku on Thursday October 19, 2017 @05:21PM (#15511651)
The JDBC driver for postgres fully supports partial updates with JSONB, though?
Re:MongoDB shares jump more than 30% in $192M IPO by collyw on Thursday October 19, 2017 @01:48PM (#15509969)
That makes complete sense, but it seems to be very rare that data is non relational in my experience.