News for hackers. Stuff that matters.
Re:Why some NoSQL DBs only let you perform transactions on a... by mooreds on Friday October 30, 2015 @08:00AM (#10477277)
This was very helpful. I am in my first project with MongoDb after a lot of experience with RDBMs and I do miss transactions. There are other things I miss more: foreign keys (dobt use dbrefs!), joins, and SQL, though.

Unfortunately this application is new, so we haven't yet seen the benefit of schemaless data changes.

Re:Why some NoSQL DBs only let you perform transactions on a... by Tenhundfeld on Friday October 30, 2015 @10:59AM (#10478198)
If this new application is something that will be in production and important to a business, I would seriously question the decision to use MongoDB over PostgreSQL.

I'm not saying it's a bad decision or that MongoDB is bad in general. I'm saying it's a decision that should not be taken lightly. If you've done the analysis and feel strongly that MongoDB's the perfect fit, then cool.

However, I think people routinely discount the operational overhead that comes with using something like MongoDB. For example, they don't think about how it's much harder to find good devops people with experience maintaining a MongoDB sharded cluster, or they don't think about what happens when the CFO wants to hook the DB up to Excel and run some queries. And I'm not even talking about the additional development complexity due to implementing integrity checks and manual transaction rollback/compensation in your application code.

Those problems can be all be solved though. It just takes more effort/money. So, you need to be sure MongoDB offers advantages to offset that.

In my opinion, schemaless data changes are not even close to offsetting the overhead, and concerns about scaling beyond MySQL or PostgreSQL are probably premature optimization. Not that those are your reasons – just reasons I often hear.

And of course, I'm making a lot of assumptions, for example that the team has experience writing & operating systems on top of an RDBMS. If the rest of the team are MongoDB wizards and you're the only one new to it, then that's a different situation.

Re:Why some NoSQL DBs only let you perform transactions on a... by Roboprog on Friday October 30, 2015 @12:01PM (#10478597)
Plus, you can make "hybrid" tables in PostgreSQL that have some fields in separate indexable primary and foreign key columns (or other uses, as needed), but remaining fields shoveled into a JSON CLOB "column".

The E/R information between tables remains exposed and enforced, but the content details can be made fungible where desired. There is an extension to include JSON fields in queries, as well, though it won't be as fast as using a real column with an index on it.

Re:Why some NoSQL DBs only let you perform transactions on a... by Rezo on Friday October 30, 2015 @08:51AM (#10477459)
I was just starting a new project and had to make a choice. I looked at the MongoDB 3.0.x release notes to see if the project has managed to improve in the last couple of years, and there are still way too many "data files are not correctly recovered following unexpected system restarts" and "race condition between inserts and checkpoints that could result in lost records" issues being fixed in each release for my liking. Different projects have different tolerances, of course.

If I was leaning towards NoSQL I would probably choose something rock solid like DynamoDB on AWS. That thing (mostly) scales by simply throwing money at it, on my last project that had tens of millions of users it worked very well, but it's definitely not cheap.

For my personal project I ended up going with PostgreSQL 9.4. The initial user experience is still as bad as I remember from years ago (after installing PostgreSQL on Ubuntu, neither the logged in user nor root have a default DB, can't even run psql, cannot create new DB's out of the box, and the tutorial(!) waxes on about "Architectural Fundamentals" instead of "Do x,y,z to install and insert a few rows of data").

With Postgres I'm able to have JSON type columns, where my unstructured documents are stored, as well as regular foreign keys, transactions and joins at the table level. So far I'm very pleased with how it's working out, I think it's a good middle ground between the flexibility of NoSQL and the reliability + querying power of SQL.

Re:Why some NoSQL DBs only let you perform transactions on a... by lobster_johnson on Friday October 30, 2015 @02:12PM (#10479476)
> after installing PostgreSQL on Ubuntu

Note that this is a mainly packaging and entirely the maintainer's fault: They typically run the database under a "postgres" user, so you'll have to do "sudo -u postgres createdb" or whatever. PostgreSQL itself doesn't care; if you install it from source and start the postmaster process as yourself, then you'll have full access right away.

That said, Debian/Ubuntu packages typically don't set up a default environment for you as a convenience. It's debatable what it should do. Create a user and database for root? Postgres maps, by default, POSIX users to database users. If you run "apt-get install postgresql-server-9.4", who should it create database users for?

Also, I'm not sure you're fair to the documentation you're referencing, which seems entirely reasoanble to me. Section 1.2 of the 94 manual's tutorial, "Architural Fundamentals", is pretty important, and it's short. But if you skip it, you'll get to the next section, "Creating a database", which tells you what need, and the next one goes into the SQL stuff. Both sections work if you've installed it from source (or Homebrew, which also runs Postgres as yourself). But it would be silly to expect this documentation to tell you how to specifically do it on Ubuntu.

Re:Why some NoSQL DBs only let you perform transactions on a... by Rezo on Friday October 30, 2015 @04:17PM (#10480281)
I respectfully disagree. After reading the "Getting Started" guide, you might reasonably believe a developer would be setup and happily coding away, but in my experience it fails at the very first steps.

Which is fine, if you want to keep a high bar, but don't be surprised when people still think the product is "harder to use" than MongoDB or whatever in 2015. Blaming maintainers does nothing for the users, take ownership of the experience! If you want to be seen as approachable and easy to use, you have to actually target people new to PostgreSQL and not write the basic guide assuming you're compiling from source on Solaris or have a dedicated site DB administrator. The magic phrase "sudo -u postgres createdb" does not appear anywhere on postgresql.org that I can see, and as a experienced user you know that's not the end of it (can I use psql as my regular user after running it? No, of course not. There's many more entirely undocumented arcane incantations left!). I claim, it is in fact impossible to get up an running with no previous knowledge if all you have access to is the postgres site. Thankfully there's Stackoverflow, so at least some users are still getting through the gauntlet.

Sorry for the rant. I get a bit frustrated when I see a great project that I love dropping users through a bad onboarding experience (I know, contributions welcome...)

Just for reference, check out: https://docs.mongodb.org/manual/tutorial/install-mongodb-on-... https://docs.mongodb.org/getting-started/node/ In fact, you get different sets of documentations for the whole matrix of OS * ENV (Shell, Node, Python, C++, Java, C#). When the docs have their target audience down to a T ("Why yes, I AM a C++ developer on OS X! This looks like the perfect fit!"), it's easy to see why they're so popular.

Re:Why some NoSQL DBs only let you perform transactions on a... by nasalgoat on Friday October 30, 2015 @08:14AM (#10477332)
At first it seems quite freeing - no schemas holding you back. Later on, after things stabilize, it stops being helpful and starts being a burden to performance and stability.
Re:Why some NoSQL DBs only let you perform transactions on a... by zurn on Friday October 30, 2015 @08:25AM (#10477377)
PostgreSQL + JSON gets you working transactions and schemaless.
Re:Why some NoSQL DBs only let you perform transactions on a... by radicalbyte on Friday October 30, 2015 @08:40AM (#10477416)
Plus it scales well and doesn't randomly lose data. Oh, and the project leaders are professionals who are honest and transparent about the capabilities of their product.
Re:Why some NoSQL DBs only let you perform transactions on a... by Rezo on Friday October 30, 2015 @08:59AM (#10477492)
It scales, to a point. In my opinion however, any type of sharding/clustering is still a total mess in PostgreSQL. Most projects will however probably never need to.
Re:Why some NoSQL DBs only let you perform transactions on a... by Roboprog on Friday October 30, 2015 @12:07PM (#10478637)
How DARE you, sir, imply that we are not the next "Amazon"!

Kidding aside, I suspect with a little discipline, one could design a PG DB that could be ported to Oracle (Exadata???) should the need, and budget ($$$$$), for that sort of scaling arise. Cheap and safe now, Expensive and safe later (if needed)

Re:Why some NoSQL DBs only let you perform transactions on a... by Roboprog on Friday October 30, 2015 @12:10PM (#10478653)
Also, I seem to recall that PG scales up on multiprocessor hardware fairly well (vs say MySQL), so one can explore exhausting that limit first before more drastic measures are taken.
Re:Why some NoSQL DBs only let you perform transactions on a... by s_kilk on Friday October 30, 2015 @11:46AM (#10478504)
Shameless plug here, but my side-project (http://bedquiltdb.github.io) aims to provide a nice mongo-a-like programmatic API on top of PostgreSQL and JSONB.

In theory there's no reason why we shouldn't be able to support a nice transactions API in the client drivers too, I just haven't gotten around to it yet.

Re:Why some NoSQL DBs only let you perform transactions on a... by joeclark77 on Friday October 30, 2015 @12:37PM (#10478844)
The idea is that you won't need database-wide transactions if your data is structured into "natural aggregates" such as orders, articles, profiles, etc. In a relational database, something as simple as an "order" has components in lots of different tables -- customer, product, order, line_item, shipping_method, etc -- so you have to lock all of those tables when doing an INSERT or UPDATE.

In the ideal MongoDB implementation, an "order" would be a document of its own, and all the other parts would be attributes or nested structures. So you only really need a transaction for the document itself. Other documents could be inserted/updated at the same time without creating a conflict. That's the theory, at least. Martin Fowler has a good Youtube on "Introduction to NoSQL" that I sometimes show to my students: https://youtu.be/qI_g07C_Q5I

The other thing about NoSQL is that it's often used for write-once, then read-only applications. Twitter, Facebook, or Instagram would be good examples. When 1% of your queries are INSERTs and 99% are SELECTS (with UPDATE and DELETE almost never occurring), you really don't run as much risk of anomalies. I would not recommend NoSQL for something like a banking application where you're doing lots of updates and need to guarantee consistency.

Re:Why some NoSQL DBs only let you perform transactions on a... by arosenbaum on Friday October 30, 2015 @03:22PM (#10479945)
>I would not recommend NoSQL for something like a banking Many of the leading banks in the world have replaced Oracle with NoSQL (generally MarkLogic) for key systems. Abadi explained quite clearly why NoSQL is not incompatible with transactions and updates.