News for hackers. Stuff that matters.

A comment on Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database. Parent comment

Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by m0zg on Sunday April 14, 2019 @02:45PM (#19660939)
I've used it briefly internally at Microsoft 2009, I had almost the same experience. Super slow, takes a while to even start the query, non-standard query language. For internal users there were even more hurdles: you had to get approval and quota before you were able to use it at all, and the default answer to both requests was "no", you really had to justify it. As an aside, at MS "no" was the answer to any request you make of Legal. Lots of bicycles were reinvented from scratch, with lots bugs, just because Legal would not approve the use of BSD or MIT licensed code.

Imagine my shock when I started at Google a year later and discovered Dremel (now BigQuery) where I could just run a query and get the results back in seconds over tens of terabytes of data, with no quotas or approvals.

I think having a system like Dremel is a huge competitive advantage, because you don't have to guess blindly about things, you can just query them and get results immediately. If Cosmos is still the best MS has internally, they need to roll their own Dremel asap. Lots of great people from SQL Server team work at Google now, but MS still has plenty of people who could do a competent petascale columnar DB with one hand tied behind their back.

[ Parent | on HN ]
Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by dtrailin on Sunday April 14, 2019 @05:38PM (#19661856)
That's impressive, considering that this CosmosDB development only started in 2014! You are talking about Cosmos which is still around but is now used as the basis of Azure Data lake [0]. However, I think the closest MS has to Dremel is actually Kusto [1] which has a custom query language but works pretty well.

[0] https://azure.microsoft.com/en-ca/solutions/data-lake/

[1] https://docs.microsoft.com/en-us/azure/kusto/query/

[ Parent | on HN ]
Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by m0zg on Sunday April 14, 2019 @06:25PM (#19662059)
You're probably right. However, having worked at MS for nearly a decade in the past, I'd be stunned if Cosmos did not serve, at least in part, as the foundation, and CosmosDB did not inherit a good chunk of the team.

Dremel now uses standard SQL and unlike Kusto it supports multiway joins and a bunch of other things expected of a more "general purpose" analytical DB.

Ultimately MS efforts will be stymied by the fact that their underlying storage story is nowhere near as good as Google's Colossus.

I won't disclose the details, but that's another thing that blew my mind: I could do linear reads (which is what columnar DBs do nearly all of the time) faster than I could process the data, from hundreds, or even thousands of workers at the same time.

The IO throughput there is truly immense, which is especially impressive given that all of the storage is remote.

[ Parent | on HN ]
Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by CurtHagenlocher on Sunday April 14, 2019 @07:38PM (#19662329)
Cosmos and CosmosDB have next-to-nothing in common and were not built by the same people.
[ Parent | on HN ]
Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by manigandham on Sunday April 14, 2019 @08:46PM (#19662550)
Why would you be stunned? They are completely separate systems.

It's like saying BigQuery and BigTable share something because they both have "Big" in their name.

[ Parent | on HN ]
Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by CurtHagenlocher on Sunday April 14, 2019 @04:02PM (#19661364)
It sounds like you're describing the internal big data platform "Cosmos", something which has been made publicly-available as "Azure Data Lake Analytics" and which has no relationship to "CosmosDB" other than a similar name.
[ Parent | on HN ]
Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by metildaa on Sunday April 14, 2019 @04:39PM (#19661565)
Sounds like a branding mishap, Microsoft created the same type of hard to understand mess with the Microsoft Dynamics branding.
[ Parent | on HN ]
Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by int_19h on Sunday April 14, 2019 @08:05PM (#19662423)
Like he said, the public product is called Azure Data Lake, and internal product names aren't brands (that's kind of the whole point of them).
[ Parent | on HN ]
Re:Azure Cosmos DB: Microsoft's Cloud-Born Globally Distribu... by curiousDog on Sunday April 14, 2019 @08:10PM (#19662438)
You're conflating CosmosDB and Cosmos (which has been publicly released as Azure Data lake Analytics and botched imo).
[ Parent | on HN ]