News for hackers. Stuff that matters.
Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by xstartup on Friday March 09, 2018 @09:41AM (#16552010)
Cassandra is JVM GC issues

Redis is in-memory only

Cloud Storage - Not sure, how we can use it outside of cloud vendors

RocksDB - Facebook just outsourced the engine to the community, where is the service which adds replication, clustering and network interface on top of it? I am sure, they use one internally, why is it not being open-sourced?

There is also badger but most of these only offer low-level operation.

Sorry, most of my developers are unable to consume them just like Redis.

Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by majidazimi on Friday March 09, 2018 @10:36AM (#16552421)
> where is the service which adds replication, clustering and network interface on top of it?

Actually I would prefer such kind of database (just the engine).

1. Expose it to network via what ever framework you like. Thrift, Rest, grpc, ... You don't have to include different kind of libraries for each network service. I would love to connect to every network service (Redis, Elasticsearch, Cassandra, MySQL, ...) via a single framework (say grpc).

2. In most large scale scenarios, there is already some kind of log service (DistributedLog, NATS, Kafka, ...). Why not take benefit of that for replication? Isn't it great to separate the engine layer from replication layer? Currently we are doing double replication actually. Replicate data from master DB to slave DB. Then replicate the same data, from any DB to cache, search, ... components. The data is already there on log. Let everyone (slave DB as well as cache/search module) consume it. This is basically state machine replication idiom. PNUTS[0], Twitter K/V database, LinkedIn Espresso [1] (as well as Ambry[2] which is their internal object store), ... use this approach for replication.

3. I would agree with that, they only support basic low level operations.

[0] http://www.vldb.org/pvldb/1/1454167.pdf

[1] http://www.csce.uark.edu/~xintaowu/BDAM/p1135-qiao.pdf

[2] http://dprg.cs.uiuc.edu/docs/SIGMOD2016-a/ambry.pdf

Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by kodablah on Friday March 09, 2018 @10:04AM (#16552180)
Agree w/ all these points (forgot "S3 has provider lock-in" though others replicate the API). I do use Cassandra for most use cases and I don't hit GC things, but I understand the concerns that come with the JVM (no, haven't subbed Scylla in yet). One that I haven't tried but want to hear the downsides of is https://github.com/pingcap/tikv (not the DB built on top, but just that one for KV). A nice published list of all cons of all database systems would be ideal.
Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by SanFranManDan on Friday March 09, 2018 @10:13AM (#16552248)
From what I could tell, its tikv is virtually useless without the rest of the tidb (the MySQL layer that sits on top written in go) to be useful.

TiKV has no replication / sharding built in, that is actually handled by a Placement Driver (PD)

From the docs: > TiKV is a component in the TiDB project, you must build and run it with TiDB and PD together.

Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by bpicolo on Friday March 09, 2018 @12:30PM (#16553285)
You can disk-back redis at various consistency levels if you so choose

https://redis.io/topics/persistence

Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by nilkn on Friday March 09, 2018 @01:29PM (#16553747)
Redis supports at least two forms of persistence: (1) periodic checkpoints; (2) writing an append-only log which can be periodically compacted to save space.
Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by electrum on Friday March 09, 2018 @02:27PM (#16554226)
> RocksDB - Facebook just outsourced the engine to the community, where is the service which adds replication, clustering and network interface on top of it?

https://myrocks.io/

Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by petre on Saturday March 10, 2018 @01:27AM (#16557395)
They also released MyRocks, a RocksDB engine for MySQL which will is being intergrated in MariaDB 10.3 and Percona Server 5.7. I don't think it supports replication yet, but it's a start.
Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by zilchers on Saturday March 10, 2018 @01:01PM (#16559435)
Cassandra has a host of other trade offs besides java (tombstones and data garbage collection come to mind).
Re:Anna: A Fast, Scalable, Flexibly Consistent Key-Value Store by mspoof on Monday March 12, 2018 @06:39AM (#16567211)
Rocks Storage engine in Cassandra which claims to reduce the GC stalls

https://engineering.instagram.com/open-sourcing-a-10x-reduct...