News for hackers. Stuff that matters.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by NickGerleman on Saturday June 10, 2017 @12:35AM (#14526177)
This is really cool but I'm curious why Bloomberg would need this. Ie, what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this. Was there nothing off the shelf that would have fit their needs?

I don't mean that in a derogatory way, I'm just curious what motivated making this.

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by raldi on Saturday June 10, 2017 @12:48AM (#14526209)
"Non-engineering company" -- ha!

But to answer your question: Mike Bloomberg's autobiography talks about this. When they started in the 80's there wasn't as much great off-the-shelf software as there is today.

Their customers were (and always have been) insanely demanding when it comes to reliability and speed. The last thing Mike wanted to do was be caught sheepishly explaining to the CIO of Merrill Lynch, "Well, gee, our Oracle database has this bug they can't fix for the next two weeks..." or even "...this optimization they can't make for the next six months"

This is a company that invented its own layer-2 network protocols 35 years ago just to squeeze every last drop of performance and reliability out of the hardware. Of course they wrote their own database (actually two -- there was a comdb 1 of of course, too)

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by apaprocki on Saturday June 10, 2017 @10:22AM (#14527428)
Just for some scale, we have 4k+ engineering employees and if we were a public company, we'd be somewhere around the 5th largest software company in the world by revenue. (Strange stat, but our primary product is software and accounts for over 3/4 revenue)
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by raldi on Saturday June 10, 2017 @04:42PM (#14529114)
Looking forward to seeing a fastsend / PRCCOM paper one day.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by apaprocki on Sunday June 11, 2017 @11:20AM (#14532257)
PRCCOM is no more! Don't need to keep bringing that one up :)
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by raldi on Sunday June 11, 2017 @12:54PM (#14532688)
Oh, in that case I hope you guys see fit to write it up one day. It's a legitimately interesting and clever solution, even if obsolete.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by magnawave on Saturday June 10, 2017 @12:48AM (#14526206)
I definitely wouldn't call Bloomberg a "non-engineering" company if you've ever worked with any of their trading / market data products. Pretty much true of any trading side finance company nowadays.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by adenadel on Saturday June 10, 2017 @12:42AM (#14526190)
Bloomberg started as a technology company. Sure, they have expanded into media, but I'm pretty sure they are still first and foremost a provider of software (plus hardware) and data.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by raldi on Saturday June 10, 2017 @12:55AM (#14526221)
I would bet that the Terminals are responsible for something like 140% of their profits, and that the media empire is a huge loss leader / vanity project.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by gaius on Sunday June 11, 2017 @10:15AM (#14532015)
The whole point of Reuters news is to inform traders about things that will lead to more trades being done over the Reuters platform, so it's entirely likely Bloomberg operates the same model.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by alexjscotti on Saturday June 10, 2017 @04:44AM (#14526641)
So. We did a few things better than the industry to this day. HA is a spectrum. Transparent masking of all failures in any state of arbitary SQL transaction is on the far right side of that spectrum.

To directly answer your question: go look at where 'replication' was on rdbms 14 years ago. PostgreSQL? MySQL?

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by ericpearl on Saturday June 10, 2017 @10:15AM (#14527401)
14 years ago Sybase had strong replication. Sybase was wall street's darling for a long time starting in the late 80's.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by 75dvtwin on Saturday June 10, 2017 @11:45AM (#14527735)
>>what special needs does Bloomberg have that would lead to a primarily non-engineering company investing the resources to create this

They probably open sourced the database, and would do more -- to dispel the above notion, and therefore to attract more qualified recruits.

Financial services companies, and technology suppliers around them, have came up much of the 'modern day' tech, some times decade(s), before google's and facebook's of the world

  -- have had NoSQL with stored procs(eg Goldman S, early to mid 90s ),   
  -- in-house grown programming lang (APL+ based in Morgan Stanley) emphasizing vector-based operations
  -- Smart contracts 
    https://www.lexifi.com/product/technology/contract-description-language
  (where a contract is represented as algorithm specified in a domain specific language) 
    ).  This was way before etherium, started, I believe, in Credit Swiss (but not sure)

   -- one of the fastest time series database (kdb+)
   -- Data science and machine learning (modeling risk and valuations)
Today's hedge funds manage petabytes of data. So do many of the big investment banks...
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by jroblak on Saturday June 10, 2017 @12:44AM (#14526197)
https://bloomberg.github.io/comdb2/overview_home.html

>> We had several goals in mind. One was being wire-format compatible with an older internal product to allow developers to migrate applications easier. Another was to have a database that could be updated anywhere and stay in sync everywhere. The first goal could only be satisfied in-house. The second was either in its infancy for open source products, or available at high cost from commercial sources.

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by foobarbazetc on Saturday June 10, 2017 @01:28AM (#14526276)
I've worked at midsize "non-engineering" companies that do boring stuff like logistics where they wrote their own RDBMS and even virtual machines for legacy hardware.

It's not that uncommon especially if you've been around a long time.

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by addicted on Saturday June 10, 2017 @10:54AM (#14527544)
Bloomberg was doing cloud computing decades before the term was even invented.

Edit: As someone else points out, they were doing cloud computing before the internet existed.

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by keeptrying on Saturday June 10, 2017 @03:25AM (#14526486)
People forget that Bloomberg had computer networks before the Internet.

I remember when we didn't even have comdb2.

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by joshu on Saturday June 10, 2017 @12:45AM (#14526198)
Bloomberg is a massive tech company.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by zerr on Saturday June 10, 2017 @04:37AM (#14526626)
If even advertising companies are engineering shops, what makes you think fintech companies aren't?
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by gaius on Sunday June 11, 2017 @06:15AM (#14531425)
Right, it's like everyone's forgot what Google's business model actually is!
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by shusson on Saturday June 10, 2017 @12:40AM (#14526184)
The work started 12 years ago when there were probably no `NewSQL` [1] solutions out there.

https://en.wikipedia.org/wiki/NewSQL

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by zeusk on Saturday June 10, 2017 @12:42AM (#14526187)
Bloomberg, after all, is a high-tech news and information delivery network.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by nemothekid on Saturday June 10, 2017 @04:39AM (#14526634)
>Was there nothing off the shelf that would have fit their needs?

I can't think of any solid distributed RDBMS that would have been around in 2004. Does anyone with more knowledge have any idea (open or not)?

Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by Keyframe on Saturday June 10, 2017 @08:38AM (#14527117)
IBM DB2 and Oracle 9i were around for sure. Other solutions as well. DB landscape was well-established back then.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by dsparkman on Saturday June 10, 2017 @10:57AM (#14527561)
They were, but rediculously expensive at the time. $100k+ at the time to have a non-replicated version on Oracle running on a multicore server. Dealing with Oracle was like dealing with the mob.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by Keyframe on Saturday June 10, 2017 @11:18AM (#14527639)
Indeed, but that's a whole other issue.
Re:Comdb2 – Bloomberg's distributed RDBMS under Apache 2 by gaius on Sunday June 11, 2017 @10:16AM (#14532019)
DEC RDB might have done it, if it was still available.