News for hackers. Stuff that matters.
Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by daxorid on Wednesday October 04, 2017 @11:33AM (#15401193)
I can't find any documentation on how, exactly, ACID is achieved across all these partitions. It's impossible to reason about performance characteristics in the absence of documentation.

For example, if they use two-phase commit, what is the granularity of locks ( table, row, etc )during an open transaction? Table locks during two-phase commit with, say, 50ms coast-to-coast latency would be an immediate dealkiller.

Or, if it's eventually consistent as most asynchronously distributed DBs are, they can't claim AC.

I really wish Cockroach had better documentation - not so much how to use it, but how it works so as to let engineers reason about performance characteristics.

Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by knz42 on Wednesday October 04, 2017 @11:49AM (#15401363)
(working for cockroach labs)

As explained in the blog (as linked by another response) and the documentation (https://www.cockroachlabs.com/docs/stable/strong-consistency... and https://www.cockroachlabs.com/docs/stable/distributed-transa...) CockroachDB does not use locks, and write intents in the concurrency control scheme are expressed by K/V pair i.e. per table cell.

Meanwhile, I acknowledge that CockroachDB does not yet convey well its performance model so that engineers can get an intuition about it. There is documentation, but it's either too detailed (like the design doc) or too high-level (most of the docs site). This needs a new sort of documentation, which we are working on.

Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by eternalban on Wednesday October 04, 2017 @12:06PM (#15401504)
From the first link:

> Employs Raft, a popular successor to Paxos.

I don't believe that is correct and that is not a good meme to promote. Successor implies the 'deprecation' of Paxos, which is certainly not the case. For the architectural sweet spot where both protocols can be used, it is fair to say Raft provides the more accessible approach, and that for those who insist on rolling their own consensus mechanism.

Paxos and Raft serve distinct architectural concerns in the shared context of providing a distributed state with formally defined consistency guarantees. And to the point, it is entirely reasonable to see a distributed system that utilizes both to address system component specific concerns.

CockroachDB, ironically (in context of your OP), in fact could benefit from a vendor lock-in were the candidate cloud provider to provide atomic clocks and GPS cards in their offerings.

Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by ddorian43 on Wednesday October 04, 2017 @12:30PM (#15401683)
What are the GPS cards for ?
Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by infogulch on Wednesday October 04, 2017 @12:39PM (#15401755)
A lot of synchronization problems are solved with access to a very accurate synchronized clock. E.g. with it you can determine a reliable "happens-before" relationship between all in flight transactions, among other useful things.

Literally all GPS does is provide a way to determine the time very accurately in relation to a bunch of synchronized atomic clocks in orbit. (As a side benefit, knowing the time like this will also give you your position.)

Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by phamilton on Wednesday October 04, 2017 @01:57PM (#15402568)
To expand:

Windows for concurrent operations are bounded by clock uncertainty. If you say you did something at 2:00 and I say I did something at 3:00, you probably did your thing before me. But if it's 1:57 vs 1:59, there's strong possibility my clock is fast or yours is slow. If I know both our clocks are within 5 seconds of the correct time, then I can say your action happened before mine. So accurate clocks allow you to provide ordering to more events.

Additionally, If you do identify operations as concurrent, you can explicitly deal with them. Dealing with them is doable, but expensive. If your clocks are in sync with a small margin of error, the rate at which you will have to do expensive things to resolve concurrent operation is low.

Google's Spanner database uses atomic clocks to have a max clock skew of 7ms. That eliminates most concurrent conflicts, and dealing with conflicts is fast because we can recognize conflict quickly. Basically, it waits 7ms on all writes to see if any other nodes have a conflicting write.

CockroachDB can operate in "spanner mode", but without atomic clocks it uses a max clock skew of 250ms. This is a much bigger window and identifying conflicts on write takes a lot longer.

CockroachDB recommends that you don't run in "spanner mode" because of the performance hit. Instead, the default mode does lazy conflict detection, where instead of waiting after all writes, it will sometimes wait to read data until the clock skew window has passed.

Here's a great read on the topic:

https://www.cockroachlabs.com/blog/living-without-atomic-clo...

Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by qaq on Wednesday October 04, 2017 @12:41PM (#15401763)
Time sync
Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by daxorid on Wednesday October 04, 2017 @11:58AM (#15401440)
Thanks!
Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by chubot on Wednesday October 04, 2017 @11:48AM (#15401350)
Not much detail here, but:

https://www.cockroachlabs.com/docs/stable/frequently-asked-q...

Last I heard CockroachDB is like "Spanner without atomic clocks". Not sure if that helps but at least the WAN latency is not an afterthought.

https://www.cockroachlabs.com/docs/stable/frequently-asked-q...

https://www.cockroachlabs.com/docs/stable/frequently-asked-q...

Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by andreimatei1 on Wednesday October 04, 2017 @11:46AM (#15401319)
Besides the blog, you can find interesting stuff in the source repo, in the technotes and RFC dirs: https://github.com/cockroachdb/cockroach/tree/master/docs/te... https://github.com/cockroachdb/cockroach/tree/master/docs/RF...

There's also a bunch of technical tech-talks linked from our company's website: https://www.cockroachlabs.com/community/tech-talks/

Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by occultist_throw on Wednesday October 04, 2017 @02:23PM (#15402912)
The trend Ive been seeing is "AP with eventual C". It comoes down to the idea that we have a planet, with datacenters in locations on said planet. Light down fiber or copper only goes so fast.

The "eventual consistency" part is simply acknowledging that we can't beat C when our databases are in North America, South America, Europe, Africa, Australia, and Asia. One-way comms takes time. 2 way comms, even more so.

The tradeoff is, that changes can come in, and 'eventually' will propagate to the rest of the DB nodes. It does sound not ideal initially, however this type of operation would allow a localDB running as AP to sync when back online.

The examples I'm deriving come from recent DBs, like GUN http://gun.js.org/#step1

Re:Avoid Vendor Lock-In Risk with Multi-Cloud Deployments by dastbe on Wednesday October 04, 2017 @03:30PM (#15403712)
cockroachdb and spanner are "CP with high probability of A". There are situations where the db (or possibly a subset, haven't read too deeply into the papers) can become unavailable due to a loss of quorum, but these situations are highly unlikely.