So, basically, https://litestream.io/ . But perhaps faster switching thanks to an explicit Raft setup? I'm not a litestream user so I'm not sure about the subtleties, but it sounds awfully similar.
That overly-simplified summary aside, I quite like the idea and I think the post does a pretty good job of selling the concept. For a lot of systems it'll scale more than well enough to handle most or all of your business even if you become abnormally successful, and the performance will be absurdly good compared to almost anything else.
I fail to see how this little bit of saving justifies all the complexity for run-of-the-mill web services that fit on one or a few servers as described in the article. The context isn't large scale services where 1ms/request saving translates to $$$, and the proposal doesn't (vertically) scale anyway.
The only thing in my comment that’s not directly based on the article is a handwavy 1ms/request saving estimate, and since they don’t provide any measurement, it’s anyone’s guess.
The guideline specifically advises to do what GP did: Instead of commenting whether or not someone read the article, to tell them that article answers their questions.
- In GitHub readme you mention etcd / consul. Is rqlite suitable for transaction processing as well ?
- I am imagining a dirt simple load balancer over two web servers. They are a crud app backed onto a database. What is the disadvantages of putting rqlite on each server compared to say having a third backend database.
[1] https://rqlite.io/docs/clustering/general-guidelines/#cluste...
[2] https://rqlite.io/docs/faq/#can-any-node-execute-a-write-req...
If you need to ensure that a given API request (which can contain multiple SQL statements) is atomically processed (all SQL statements succeed or none do) that is supported however [1]. That's why I think of rqlite as closer to the kind of use cases that etcd and Consul support, rather than something like Postgres -- though some people have replaced their use of Postgres with rqlite! [2]
The main caveat here is that rqlite is an out-of-process database, which you communicate with over http. That puts it on similar grounds as e.g. postgres, just significantly lighter weight, and somewhat biased in favor of running it locally on every machine that needs the data.
So minimum read latency is likely much lower than postgres, but it's still noticeable when compared to in-process stuff, and you lose other benefits of in-process sqlite, like trivial extensibility.
But if the blogger learned SQLite, how would they have a topic to blog about?
Also, no benchmarks. It's quite odd that an argument grounded on performance claims does not bother to put out any hard data comparing the output of this project. I'm talking about basic things like how does this contrived custom ad-hoc setup compare with vanilla, out-of-the-box SQLite deployment? Which one performs worse and by how much? How does the performance difference reflect in request times and infrastructure cost? Does it actually pay off to replace the dozen lines of code of on boarding SQLite with a custom, in-development, ad-hoc setup? I mean, I get the weekend personal project vibe of this blog post, but if this is supposed to be a production-minded project then step zero would have been a performance test on the default solution. Where is it?
I probably did a bad job then, because everything in the blog post was meant to be developer productivity claims, not performance claims. (I come from a developer productivity background. I'm decent at performance stuff, but it's not what excites me, since for most companies my size performance is not critical as long as it scales.)
The OP starts out by talking about periodically dumping everything in RAM to disk. I’d say that’s your checkpointing.
Just try and cold-start your database and run a fairly large select twice.
Even if you weren't using erlang, his influence (and in general, ericsson) permeates the telecom industry.
There are libraries available to wrap your stuff with this algorithm, and the benefit is that you write your server like it would run on a single machine, and then when launching it in prod across multiple, everything just works.
A transactional database is simple in Expand and Extract, but adds additional overhead during the Explore phase, because you're focusing on infrastructure issues rather than product. Data reliability isn't critical in the Explore phase either, because you just don't have customers, so you just don't have data.
Having everything in memory with bknr.datastore (without replication) is simple in the Explore phase, but once you get to Expand phase it adds operational overhead to make sure that data is consistent.
But by the time I've reached the Expand phase, I've already proven my product and I've already written a bunch of code. Rewriting it with a transactional database doesn't make sense, and it's easier to just add replication on top of it with Raft.
Unless, of course, your startup is in the business of selling DBMSes.
I think sql db make the most sense during the explore phase and you switch off of them once you know you need an improvement somewhere (like latency or horizontal scalability).
And this comes to the difference between Explore phase and Expand phase.
In the Explore phase, data migration was just running code on the production server via a REPL. Some migrations such as adding/removing fields are just a matter of hot-reloading code in CL, so there weren't a lot of migrations that we had to manually run.
In the Expand phase, once you add replication, this does become hard and we did roll out our own migration framework. But by this point we already have a lot of code built out, so we weren't going to replace it with a transactional database.
Essentially, we optimized for the Explore phase, and "dealt with the consequences" in the Expand phase (but the consequences aren't as bad as you might think).
Then you get used to near-zero latency that in-RAM data gives you, and when it outgrows your RAM, it's a pain in the butt to move it to disk :)
If your load fits entirely on one server, then just run the database on that damn server and forget about “special architectures to reduce round-trips to your database”. If your data fits entirely in RAM, then use a ramdisk for the database if you want, and replicate it to permanent storage with standard tools. Now that’s actually simple.