News for hackers. Stuff that matters.

free-ekanayaka

Re:Dqlite – High-Availability SQLite on Friday August 30, 2019 @11:16AM (#20839413)
It's not a buzz term. It's really fully async disk I/O. Dqlite does not use SQLite's stock vfs implementation for writing to disk, as it's an entirely different model (based on raft).
Re:Dqlite – High-Availability SQLite on Friday August 30, 2019 @11:14AM (#20839400)
Everything you said is very accurate (dqlite author here).
Re:Dqlite – High-Availability SQLite on Friday August 30, 2019 @11:13AM (#20839388)
Give us time, we'll get there :)
Re:Dqlite – High-Availability SQLite on Friday August 30, 2019 @11:13AM (#20839383)
Depends on how fast is your disk, what file system and kernel you use, and how low is your network latency. Difficult to predict. But it's basically as fast as it can get given 1) hardware constraints 2) raft consensus.

If you want light-speed insert/delete, you could probably don't use the disk at all: as long as a majority of your nodes don't die, you won't lose any data. You can also go somewhere in between and save to disk only at specific intervals.

Re:Dqlite – High-Availability SQLite on Friday August 30, 2019 @11:10AM (#20839354)
Not that I know, but it's on my todo list (dqlite author here).
Re:Dqlite – High-Availability SQLite on Friday August 30, 2019 @11:09AM (#20839341)
The support of async disk I/O in Linux differs depending on kernel version and file system type. But it is possible to get 100% async I/O with the is_submit(), and dqlite will leverage that if detected.

There is now a new async I/O API available in Linux (I'm not remembering the name right now, but it was developed by folks at Facebook). It looks promising so I'll check it at some point. (dqlite author here)

Re:Dqlite – High-Availability SQLite on Friday August 30, 2019 @05:20AM (#20836996)
It has been around for 2 years and half, but we released v1.0.0 just yesterday.

I added answers to your other questions to the FAQ:

https://github.com/canonical/dqlite/blob/master/doc/faq.md#w...

https://github.com/canonical/dqlite/blob/master/doc/faq.md#w...

https://github.com/canonical/dqlite/blob/master/doc/faq.md#i...

Re:Dqlite – High-Availability SQLite on Friday August 30, 2019 @04:58AM (#20836895)
Oh I had no idea somebody was using it in the wild! It has been unstable until now, we just released v1.0.0 yesterday. So no more public API breakage from now on.
Re:Distributed SQLite for Go applications on Friday April 06, 2018 @04:10PM (#16776942)
Yes, dqlite is currently used for clustering support in LXD 3.0: https://github.com/lxc/lxd
Re:Distributed SQLite for Go applications on Friday April 06, 2018 @10:43AM (#16773573)
If you read the raft paper, or just watch presentations of it, you'll see that all the issues you mentioned are covered:

- faulty node replacement: you can take any node off (or any node can crash) at any time. As long as there are enough nodes left to reach a quorum, your system will be available. If there are not enough nodes left, your system will be unavailable (but keep consistency)

- re-syncs, snapshots and the rest are all covered in the raft paper

- clients wanting to perform writes always talk to the node that is currently the leader, that node fails, clients will look for next leader (which will be eventually elected as long as there's a quorum of surviving nodes)

- overwhelming a system is a different concern, raft writes are serialized so the goal is usually not throughput (for that you might look at AP/AC storage solutions in the CAP spectrum, raft is CP). Designs that need high write throughput might still use raft as internal building block for coordination.

Re:Distributed SQLite for Go applications on Friday April 06, 2018 @06:44AM (#16772248)
"slower" means this:

when you perform a SQLite write, SQLite writes a new page on disk to the its write-ahead log. dqlite needs to replicate that page write across all nodes. A page is typically 4kb, so when you do a write on the leader node you need to transfer 4kb across the wire to all other follower nodes and wait for a quorum of them to come back and say "I got it" (which includes writing the page to disk). On the contrary rqlite just needs to transfer and store the SQL text, which is typically less than 4kb.

In practice it's still pretty performant, but I'll publish benchmarks later on.

I plan to submit the patch to SQLite upstream too, yes.

Re:Distributed SQLite for Go applications on Friday April 06, 2018 @05:31AM (#16772013)
Unit and fuzzy-based test coverage that includes all possible edge cases is being put in place. I'd like to write a Jepsen test suite too.

Hardware does not matter that much, since SQLite is pretty hardened for handling any possible hardware failure (out of memory, out of disk space, memory/disk corruption) and propagate that to client code (including dqlite). Regarding network failures (hardware-related or not), raft guards you against them, and it's formally proved.

Re:Distributed SQLite for Go applications on Friday April 06, 2018 @05:25AM (#16771990)
If you require SQL, sharding and concurrent writes, yes do use CockroachDB or other equivalent solution.

If you don't have those requirements, dqlite or rqlite might get the job done with less moving parts and operational overhead.

It's pretty much the same argument of using SQLite vs (say) PostgreSQL, but translated into distributed systems. See:

https://www.sqlite.org/whentouse.html

Re:Distributed SQLite for Go applications on Friday April 06, 2018 @02:31AM (#16771343)
I'm working towards a 1.0 release. At that point documentation will be more complete and the plumbing needed to use it in an app should be also reduced.

I think the best explanation of why you'd use it over PG or MySQL is the same as why you'd use SQLite over them:

https://www.sqlite.org/whentouse.html

With dqlite you also get HA, fault-tolerance and transparent failover, if you're willing to trade it with a bit of performance.

Re:Distributed SQLite for Go applications on Friday April 06, 2018 @02:26AM (#16771316)
BedrockDB requires to operate a separate process, whereas you can embed dqlite in your Go application (pretty much in the SQLite philosophy). Also, afaik BedrockDB patches upstream SQLite with some more intrusive changes than dqlite (e.g. for supporting concurrent writes). The SQLite patch that dqlite requires is pretty minimal and just adds hooks to internal WAL events.
Re:Distributed SQLite for Go applications on Friday April 06, 2018 @02:23AM (#16771302)
If your application is:

1) Distributed (e.g. n equal processes running on n machines) 2) Needs some shared state across nodes 3) Would like that state to have SQL semantics (relations, transactions, etc.) 4) Not to heavy on writes to this shared state (I might provide some ballpark numbers at some point) 5) Wants to avoid the operational overhead of an external storage system (mysql, postgresql) 6) Wants to be fault-tolerant and have transparent failovers

then dqlite might be a good choice.

Re:Distributed SQLite for Go applications on Friday April 06, 2018 @02:17AM (#16771279)
I implemented dqlite in Go for convenience (a production-ready raft library was available, github.com/hashicorp/raft). It'd be interesting to implement something similar in C or event batter in rust, so you can link the library from languages than just Go. But not sure there'd be good use cases for that.
Re:Distributed SQLite for Go applications on Friday April 06, 2018 @02:13AM (#16771263)
Hello, author of dqlite here.

Things are still in flux, but yes, I plan to publish benchmarks before making a 1.0 release, as well as improving documentations and introduce some more abstractions to make it easier to use.

The reason it might be slower is some cases is that RQlite replicates statements and dqlite replicates WAL frames (where "frames" roughly means "disk pages"), which are typically bigger.

I didn't run benchmarks against RQlite yet, but I'd expect performance differences to be negligible for most use cases.