News for hackers. Stuff that matters.
Re:Why we lost Uber as a user by vegabook on Monday August 01, 2016 @09:51AM (#12202334)
> Given that, there are only four dbs worth using: Postgre, Oracle, DB2, and MS SQL

This is total bullshit. Even a modest size database (3-5 terabyte) for a small company like mine, chokes a relational database bigtime on time series. Especially the modern use cases where you are ingesting fast. You need Cassandra or Hbase which are very serious pieces of technology and are definitely "worth using", and will crush any of the four relational dinosaurs you mention in that space at comparable cost, and with embarrassingly better scalability. And by the way, that space is not some corner case, it is probably the major growth area of the next computing era.

Re:Why we lost Uber as a user by PeCaN on Monday August 01, 2016 @12:59PM (#12203994)
There are column-oriented relational databases for that. DB2 and SQL Server both implement columnar systems for OLAP-esque workloads pretty well, and I think Oracle has something like that as well. In the Free software space, MonetDB is pretty good and I believe there's some Postgres extension to store data column-wise (which is probably not competitive with ground-up column stores like MonetDB, but beats row-wise storage for certain workloads).

You don't need to give up proven relational databases for this web-scale bullshit. Financial companies have been doing hundreds of times “modern use cases where you are ingesting fast” for decades. KDB+ is the darling of that industry, but DB2 is pretty popular (IIRC) as well as some other niche options.

Right tool for the right job, basically. I have nothing against Cassandra or HBase, and they're both quite impressive pieces of engineering. But they're not an alternative to relational databases; rather, a complement for very niche use-cases. I see NoSQL stuff get used for “performance and scalability” in places where a columnar relational DB would scale just fine and bring the benefits of a relational database. I'm all for HBase for non-relational use-cases, but those are rather more rare than people seem to think (and end up with an ad hoc relational model because of it).

Re:Why we lost Uber as a user by jacques_chester on Monday August 01, 2016 @05:00PM (#12205919)
For columnar and massively-parallel workloads, Greenplum was opensourced this year[0]. It was originally forked from PostgreSQL 8.2.

There's also Apache HAWQ (incubating), which takes Greenplum's SQL parser and distributed query planner to use as front-ends for Hadoop[1].

Disclaimer: I work for Pivotal, which opensourced these systems.

[0] http://greenplum.org/

[1] http://hawq.incubator.apache.org/

Re:Why we lost Uber as a user by PeCaN on Monday August 01, 2016 @05:09PM (#12206010)
Whoa, that looks quite slick. Thanks for your work!
Re:Why we lost Uber as a user by jacques_chester on Monday August 01, 2016 @05:15PM (#12206058)
All credit belongs to my peers in the Big Data division, and the engineering directors who successfully argued to opensource our entire portfolio of data tech (Greenplum, HAWQ, Gemfire and MADLib).
Re:Why we lost Uber as a user by vegabook on Tuesday August 02, 2016 @03:54AM (#12208407)
who said they were an alternative to relational databases? The problem with the post was not that the 4 mentioned were bad, only that they were the "only four worth using" which is an eye-rollingly ridiculous statement. Anybody who asserts that relational databases are the only serious databases is plain wrong, or has an ulterior motive. And this is not for "web scale bullshit" by the way. It is about large amounts of financial timeseries ingest for machine learning optimizations that I can promise you, has nothing to do with the web.
Re:Why we lost Uber as a user by Roboprog on Monday August 01, 2016 @11:55AM (#12203404)
Suggested edit: s/This is total bullshit/Perhaps you overlooked something:/.

Otherwise, this would have been a good comment, I think.

Re:Why we lost Uber as a user by Grishnakh on Monday August 01, 2016 @12:29PM (#12203717)
This is one thing I like about HN's system: it lets me edit posts for a limited period. I'll sometimes bang out a post like the above, with a rather negative opening line like that and hit "submit". Then a minute later I'll realize I need to tone that down some, so I'll go edit it, as you suggest here. Unfortunately some shitty web forums don't allow you to edit posts at all, but I do appreciate ones which do allow editing stuff like this before anyone has a chance to read it.
Re:Why we lost Uber as a user by chc on Monday August 01, 2016 @06:27PM (#12206547)
HN also lets you set a delay before a comment appears, if you're like me and tend to catch mistakes more easily on the page than in the composition box. It's called "delay" on your settings page.
Re:Why we lost Uber as a user by jamespo on Monday August 01, 2016 @03:46PM (#12205316)
I think there's something to be said for the Linus style
Re:Why we lost Uber as a user by PeCaN on Monday August 01, 2016 @05:07PM (#12205997)
Yeah, but generally only when you're actually correct. If you're not, you look like an idiot. Apparently the parent hasn't heard of column-store relational databases (which are kinda great for time series).

(I'm most familiar with SQL Server; DB2 and Oracle apparently have similar functionality.)

Apparently their 3-5TB time series table chokes ‘dinosaur’ relational databases. For a time series table you should almost certain be using a column-store index with CREATE CLUSTERED COLUMNSTORE INDEX. Depending on how much they're querying vs inserting, that alone could prevent SQL Server from choking. If it's more insertion-heavy, but they still need to run a lot of ad-hoc queries, SQL Server 2016 supports using a nonclustered column-store index alongside a row-store. You can insert with the row-store and query with the column-store. If it's still choking with that, it might be time to check out the database structure and see what's up. One of the more common killers is putting a lot of data in the time series table. Generally it's more efficient to have a clustered column-store index table that contains the timestamp and ids to metadata, and keep the metadata itself in row-store tables with appropriate indexes.

Re:Why we lost Uber as a user by vegabook on Tuesday August 02, 2016 @02:54AM (#12208228)
of course I've heard of column store relational databases, but they're not nearly as efficient nor scalable as Cassandra which has been built from the ground up for this. Fact is, if you don't need referential integrity, there is absolutely no need to pay the massive complexity overhead and horizontal scalability penalties that the relational databases incur.

Separately it is absolutely correct that a comment that asserts that 4 relational databases are the only serious ones is completely misguided and deserves a robust rebuke.

Re:Why we lost Uber as a user by qwertyuiop924 on Monday August 01, 2016 @04:53PM (#12205852)
I consider your argument misinformed.

Technology moves fast. Very fast. If a piece of tech gets old enough to be called a dinosaur, and is still run at brand-new companies, it's doing something right, even if it's not for your usecase.

And the fact is, not all of us have 3-5TB time seriesdatabases. And time series data can cause a lot of DBs to choke.

The point is, I'm glad Cassandra and Hbase, or a dedicated TSDB, or whatever, work for you. Your usecase is not the same as the rest of us, and assuming that it is is causing you to come to some incorrect conclusions.

Re:Why we lost Uber as a user by vegabook on Tuesday August 02, 2016 @03:34AM (#12208346)
Clearly the relational databases are very good and very useful in more cases than time series. However timeseries is huge, and growing - it's not a niche case anymore. Anyhow my beef was with the assertion that the 4 relational databases listed were the "only serious ones". This is completely untrue.
Re:Why we lost Uber as a user by qwertyuiop924 on Tuesday August 02, 2016 @06:54AM (#12208986)
You may have missed the line "In that case" :-)