News for hackers. Stuff that matters.
Re:Databases in 2022: A Year in Review by nikita on Monday January 02, 2023 @12:50PM (#34221061)
Neon CEO here.

There are a number of ideas in the database space that the industry is adopting across the board:

- Separation of storage and compute (Neon, AlloyDB, Aurora). Every cloud database should built one. It's a big undertaking, but benefits are undeniable.

- Good query processor for analytics (Snowflake, Velox, Singlestore)

- Open source. Especially in OLTP open source == trust

- HTAP. Can run mixed workload (Singlestore, Unistore): both OLTP (apps) and OLAP (reporting). This has always been a dream, but we still live in the world of dedicated systems: E.g. Snowflake and Postgres.

- Shared nothing sharding (Vitess). This is the most controversial as you lose compatibility with the mothership (MySQL for Vitess). So it's unclear this will be the dominant architecture in the future. I think the world may get to "dynamic sharding" where storage stays separate and compute can be multinode and the user can easily and instantly change the number of nodes.

Re:Databases in 2022: A Year in Review by gavinray on Monday January 02, 2023 @02:14PM (#34222298)
Really enjoyed the talk from your engineer Heikki at Andy's CMU "Databases!" seminar series earlier this year, great stuff =)

https://www.youtube.com/watch?v=rES0yzeERns

Also, it's been neat to have mostly worked our replacing our old Heroku integration at Hasura with you folks.

Re:Databases in 2022: A Year in Review by emptysea on Monday January 02, 2023 @02:39PM (#34222599)
I’m not well versed in the tech but is Spanner the sort of dynamic sharding approach your talking about?
Re:Databases in 2022: A Year in Review by gavinray on Monday January 02, 2023 @03:46PM (#34223418)
Scylla also uses a shared-nothing sharding approach, they have an open-source "asynchronous I/O engine" toolkit that is used inside of ScyllaDB called Seastar

They have some good reading on this approach in the Seastar docs if you're curious, it's a great general introduction to Shared Nothing:

https://seastar.io/shared-nothing/#:~:text=The%20Seastar%20M....

https://www.linuxfoundation.org/webinars/under-the-hood-of-a...

This is usually generalized to some variation of "Physical-OS-thread-per-CPU-core" architecture, where each CPU core gets it's own dedicated chunk of data. You use hashing or some other mechanism to route requests for data to the right core.

Each CPU core can have user-land concurrency or "virtual threads" (fibers/coroutines/etc) as well, so you're not limited to sequential code necessarily.

A combination of thread-per-core x userland-coroutines-per-thread is how modern query processing approaches like "Morsel-Driven Parallelism" work.

  CPU-1-OS-THREAD-1
    COROUTINE_1
    COROUTINE_2
  CPU-2-OS-THREAD-2
   COROUTINE_35
   ...
   COROUTINE_102
https://db.in.tum.de/~leis/papers/morsels.pdf
Re:Databases in 2022: A Year in Review by AtlasBarfed on Tuesday January 03, 2023 @01:08AM (#34228313)
Scylla is a C++ rewrite of cassandra, the threading has been better as well with thread-per-core. Cassandra 4.0 has some thread-per-core rearchitecture so it remains to be seen if Scylla has performance advantages it used to tout.

I have not checked if Scylla ever caught up to Cassandra 3.0's feature set, last I checked it had not. It may be "going its own way" on features as well. I believe (they can correct me).

My personal recommendation would be to start with Cassandra, and when operations allow it, move to scylla if performance testing shows it is better. There is also exotic stuff like Rocksandra. Cassandra and Scylla are fundamentally similar in operations (add/remove nodes, scaling, etc). The scylla folks will contend the operations are easier since they get more perf per node so you need less nodes.

It's a good idea with cassandra / scylla to bootstrap with a performance testing, data loading, provisioning loop. It gets you used to operations and the tooling for backups, loads, and load testing and metrics.

Re:Databases in 2022: A Year in Review by samsquire on Tuesday January 03, 2023 @01:51AM (#34228611)
Thanks for this comment Gavin.

I use colon separated syntax to indicate the multiplexing of different hierarchy levels of thread/coroutine/lightweight threads.

1:M:N means 1 scheduler thread, M kernel threads and N lightweight threads. So 8 lightweight threads multiplexed over 2 kernel threads multiplexed by one scheduler thread looks as this:

  1:0:0
  1:0:1
  1:0:2
  1:0:3
  1:1:0
  1:1:1
  1:1:2
  1:1:3
Can use ringbuffers to route data between threads, there's a multiproducer multiconsumer ringbuffers available. I also wrote what I call token ring parallelism where there is no locks but threads take turn writing but read as fast as they can.

My token ring parallelism can be found in this sourcefile https://github.com/samsquire/multiversion-concurrency-contro...

Essentially there is a state machine that each thread is in reading mode or writing mode. Each thread enables the writing mode of the next thread THREAD_ID + 1 % threads.size(). When the last thread is reached, writing mode is enabled and then when that finishes, reading mode for all threads is reenabled.

Need some approach to representing data flow between threads that is efficient and is pull/push driven.

Re:Databases in 2022: A Year in Review by jandrewrogers on Monday January 02, 2023 @02:50PM (#34222750)
The separation of storage and compute is controversial, though it is definitely convenient for cloud operators. It is literally going backward in terms of the available storage bandwidth, and many modern database architectures can very effectively use all the storage bandwidth available in modern systems. There is not enough network bandwidth, and won’t be for the foreseeable future, for running storage over networks to be efficient. Database workloads are also starting to move to edge environments, which make it even worse. The resource mismatch currently leaves a lot of compute sitting idle in practice.

Another issue with this model is that it tends to offer poor ratios of bandwidth to storage size. This is at odds with the desire to run mixed workloads in a single system.

Re:Databases in 2022: A Year in Review by gavinray on Monday January 02, 2023 @03:35PM (#34223296)

  > The separation of storage and compute is controversial
I don't think it's a controversial topic, it's an emerging trend. See for example this slide from the recent RocksDB meetup on their vision for the next 3 years of RocksDB's future:

https://twitter.com/GavinRayDev/status/1600666127025156096

Disaggregated storage is becoming quite a hot topic, and there are several technologies fueling this.

There are two major advances happening in terms of disk I/O right now (that aren't to do with async IO uring stuff)

1. NVMe Zones + NVMe FDP specs, which are ways for userland applications to control placement of writes on drive (like a software Flash-Translation-Layer)

2. NVMe-over-Fabrics or similar technologies (with rdma/tcp) enables the access of a remote disk with 'comparable' latency to that of a local one.

E.g. if local disk access is 100us, AWS can give you 110us with NVMe-OF.

These two advances support the growing trend of disaggregated storage

---

Full credit to Arun George from the Samsung Memory group for explaining this to me.

Re:Databases in 2022: A Year in Review by jandrewrogers on Monday January 02, 2023 @04:28PM (#34223910)
The storage-over-fabric latency improvements allow it to work well for software that was not particularly optimized for storage performance, which has obvious benefits that I am not arguing. There is a good reason storage-over-fabric has been a popular database deployment model for decades, it does have advantages, but this emerging trend is nothing new nor are the issues that it has.

For systems that are highly optimized for storage performance, storage bandwidth tends to be more important than storage latency and that is what is being sacrificed for the sake of disaggregating storage. I see integer factor reductions in workload throughput when deployed on disaggregated storage versus the usually NVMe JBOD, and that factor tracks real-world storage bandwidth pretty well, particularly at high storage densities. At scale, these economics start to matter quite a lot.

To be clear, disaggregated storage has no effect on database design generally, seeing as how it has been widely used for decades. It does put an artificial floor on achievable throughput for many workloads due to the bandwidth bottleneck. When people realize this then the pendulum swings the other way (again), so I wouldn't want to overfit for disaggregated storage.

Re:Databases in 2022: A Year in Review by gavinray on Monday January 02, 2023 @04:35PM (#34223989)
I didn't realize who I was talking to, just checked the username

Would encourage other folks to do the same/read post history

Re:Databases in 2022: A Year in Review by sitkack on Monday January 02, 2023 @05:43PM (#34224736)
The truthiness of your comments shouldn't change upon who hears them?
Re:Databases in 2022: A Year in Review by gavinray on Monday January 02, 2023 @05:51PM (#34224819)
No, I still stand by my opinion that I don't find disaggregated storage controversial

But if you're going to take an opinion on databases from someone, jandrewrogers is probably a good person to take them from. Me, on the other hand, I'm just a hobbyist/enthusiast, I've no accolades under my belt and my experience on the engineering end is shy of a year.

Re:Databases in 2022: A Year in Review by sitkack on Monday January 02, 2023 @07:25PM (#34225767)
> controversial

You and jandrewrogers are using different connotations for this word. Separating compute and storage makes sense when you have already done this like the hyperscalers have. It is good for them but not necessarily for the end user.

That it is surprising to you are not doesn't say whether it is a good idea or not. If you make a statement regarding your own opinion vs an expert, then that statement is not referentially transparent, why?

When disaggregated storage gets predicate pushdown will it still be disaggregated?

Re:Databases in 2022: A Year in Review by glogla on Monday January 02, 2023 @03:52PM (#34223487)
The separation of compute and storage is most of all convenient for whoever operates the database.

Need more compute, or scale it down when not in use? You can add it in minutes, no re-balancing, no careful moving of data. Need more storage? You have it on tap. Want to give some group of users their dedicated compute? Or run differently configured computes on top of one data, including even using different compute engines for batch and for interactive? You can.

Compare Snowflake "create warehouse", which can give you compute that can immediately access any data, with a dance you need to do to resize Clickhouse and decide which tables are replicated in what way.

At the same time, everything you say is true, and that is why system without storage-compute separation can give you order of magnitude more performance for the same money than a separated system can. If someone could solve the fragility and hassle of it, it would be quite a winner.

Re:Databases in 2022: A Year in Review by AdamProut on Monday January 02, 2023 @05:27PM (#34224550)
There are hybrid designs for separation of storage and compute that are aimed at mixed workloads[1]. They avoid writes to remote storage on transaction commit (i.e., act like a shared nothing databases for commits, but still push data asynchronously to a shared remote disk that can be used for scale up/point in time restores/branching).

[1] https://www.singlestore.com/blog/separating-storage-and-comp...

(disclosure: SingleStoreDB CTO)

Re:Databases in 2022: A Year in Review by nikita on Monday January 02, 2023 @05:38PM (#34224686)
Adam, you still synchronously replicate each log record to 2 places, right? Technically this should be roughly equivalent to writing into a consensus. And use write through cache for reads.
Re:Databases in 2022: A Year in Review by AdamProut on Monday January 02, 2023 @05:48PM (#34224794)
Yep, for writes network bandwidth usage is independent of separation of storage of compute in some sense. Any database that provides high availability is writing over the network somewhere before it acks a transaction committed. It matters who is on the other end of that network write though. Take the typical Cloud DW (i.e., Snowflake) design of forcing writes to the blob storage (shared remote disks) before commit. That is a much higher latency write then what a high performance transaction log replication protocol will do to replicate a write to another host.
Re:Databases in 2022: A Year in Review by zX41ZdbW on Monday January 02, 2023 @07:36PM (#34225890)
SingleStore is a kind of uninteresting example because it is not open-source. It makes potential customers feel non-confident. Why move large data volumes to SingleStore if the company can cease to exist in a year?
Re:Databases in 2022: A Year in Review by AdamProut on Monday January 02, 2023 @09:26PM (#34226777)
Yep, SingleStoreDB (formerly MemSQL) is not open source (probably never will be), but it does have many paying customers who have had workloads in production for over a decade at this point.

Also, isn't taking a bet on a very recently launched database as a service based around an open source database also pretty risky? Say for example the very recently launched Clickhouse Inc. service (which your a co-founder of?).

Re:Databases in 2022: A Year in Review by zX41ZdbW on Monday January 02, 2023 @09:47PM (#34226934)
Touché.
Re:Databases in 2022: A Year in Review by password4321 on Tuesday January 03, 2023 @12:11AM (#34227951)
I'm interested to see when SingleStore will support ARM.

https://www.singlestore.com/forum/t/singlestore-for-arm64-m1...

Re:Databases in 2022: A Year in Review by nikita on Monday January 02, 2023 @05:46PM (#34224771)
> The resource mismatch currently leaves a lot of compute sitting idle in practice.

When separate storage is multi tenant and compute is elastic you can really squeeze more efficiency. With that you can keep utilization extremely high, b/c storage is effectively spread across the whole fleet. We really see this now running 10K databases on a single (single per region really) storage deployment.

Compute is also elastic, so you again can drive utilization up. So my argument is that separation of storage and compute is not necessarily for performance but rather for utilization, convenience to operate, and additional capabilities (branching, serverless).

Re:Databases in 2022: A Year in Review by jandrewrogers on Monday January 02, 2023 @10:59PM (#34227473)
Sure, for aggregating many small databases this makes sense. Disaggregation of storage and compute has been popular at many times in database history, the tradeoffs are well-understood. Some databases are not particularly performance sensitive and it eliminates resource waste from resource overhang when databases only need fractions of server. For single large databases that occupy many servers, the economics tend to follow the size of the cluster which can be significantly smaller with direct-attached storage. Scaling storage independent of compute introduces some interesting edge cases operationally because they are intrinsically coupled to some extent — scaling one creates resource pressures and bottlenecks elsewhere.

An alternative model to disaggregation that can produce a similar dynamic balancing of compute to storage without sacrificing bandwidth in scale-out systems is to use a dynamic mix of heterogeneous nodes. The storage is still tightly coupled to the compute but the ratio of compute to storage of the aggregate system can be quickly adjusted on the fly by adjusting the mix of server types. I haven’t seen a lot of work on heterogeneous cluster architectures in a long time — it was impractical when database clusters were capex — but it is a proven model albeit complex to implement, and easier to deploy today thanks to the cloud.

Many of the emerging data problems at the edge look like heterogeneous cluster architecture problems too if you squint, so it likely has value beyond the data center. The edge has a lot of surprisingly-shaped problems that don’t fit any of our current tools e.g. some of the most extreme scale-up database problems I’ve seen in any context.

Re:Databases in 2022: A Year in Review by unusualmonkey on Tuesday January 03, 2023 @12:12AM (#34227958)
I've used BigQuery extensively in the past. Have to say network bandwidth didn't seem a problem when querying terabytes of arbitrary data in seconds.

What problems do you forsee?

Re:Databases in 2022: A Year in Review by zX41ZdbW on Monday January 02, 2023 @03:26PM (#34223186)
The most developed analytical query engine as of today is ClickHouse [1].

It is open-source, has the separation of storage and compute and the best performance.

ClickHouse is a source of inspiration for many emerging database engines: Velox, Doris, DuckDB, Datafusion...

[1] https://github.com/ClickHouse/ClickHouse

Disclaimer: I work on ClickHouse.

Re:Databases in 2022: A Year in Review by ddorian43 on Monday January 02, 2023 @04:38PM (#34224018)
> has the separation of storage and compute and the best performance.

This part is not yet production ready though. And it's unclear when it will be. Though no other open source OLAP does it either AFAIK.

Re:Databases in 2022: A Year in Review by _dark_matter_ on Monday January 02, 2023 @09:09PM (#34226649)
SparkSQL + delta lake definitely counts as an open source OLAP separated compute/storage.
Re:Databases in 2022: A Year in Review by swyx on Monday January 02, 2023 @05:14PM (#34224411)
since you were also a founder of Singlestore, just wondering - what will be Neon's take on HTAP workloads?

trying to see how much you mean it that the industry is "adopting (HTAP) across the board"

Re:Databases in 2022: A Year in Review by nikita on Monday January 02, 2023 @05:33PM (#34224638)
I wouldn't say HTAP is an across the board trend. The need is mostly in the enterprise where we see high value mixed workloads.

Neon will take a Postgres centric POV. I consider a few approaches, but not committing to any particular at this point. The reason to not commit is that there is a lot more opportunity for Neon in serverless, devtools, and edge. And it makes a ton more sense to partner for OLAP workloads. So what is Postgres centric POV:

- Plugins like Timescale and Citus

- Seamless integration with sending CDC to Snowflake, Singlestore, Databricks, Clickhouse, and Motherduck

- FDW for integration with Snowflake, Singlestore, Clickhouse, Databricks, and Motherduck

- We looked at putting DuckDB on Neon storage with Jordan Tigani, but it was too big of a lift as DuckDB doesn't push all data through the WAL. Maybe in the future.

Re:Databases in 2022: A Year in Review by zX41ZdbW on Monday January 02, 2023 @06:46PM (#34225401)
Let's commit this to ClickBench: https://github.com/neondatabase/neon/pull/2855 So, NeonDB can be compared with other serverless Postgres offerings.
Re:Databases in 2022: A Year in Review by nikita on Monday January 02, 2023 @06:50PM (#34225436)
The reason we run OLAP bench is because when you separate storage full table get impacted: you fetch pages from remote storage one by one and the mitigation is prefetch. So our goal is to be on par with vanilla Postgres. We don't care about winning OLAP benchmarks.
Re:Databases in 2022: A Year in Review by zX41ZdbW on Monday January 02, 2023 @07:33PM (#34225857)
ClickBench is still ok to compare non-OLAP databases on OLAP workloads - filter away the column-oriented DBMS, and you will be left with the comparison of non-OLAP databases.

I think it makes sense to add NeonDB for the sake of completeness.

Re:Databases in 2022: A Year in Review by nikita on Monday January 02, 2023 @09:50PM (#34226954)
I think lots and lots of benchmarks are useful internally. They highlight where the soft spots are. This is however very different from publishing benchmark results.

As a dbaas vendor we should publish benchmarks that your user base really cares about which demonstrate that the system is mature and can handle the core workload your users expect you to handle. That is why publishing an OLAP benchmark results for an OLTP system makes very very little sense.

(BTW OLAP systems like Clickhouse should really publish TPC-DS SF1000+ results)

For OLTP systems there is TPC-C but it's actually not that telling as it is so simple. I can start seeing that people in OLTP care more and more about latencies, hence all the edge related work and optimization of the drivers.

So Neon will be publishing end-to-end latencies benchmarks and also make sure it is on par with vanilla Postgres on a broad set of benchmarks. Those we will run internally and the results will simply be commits on our github.

Re:Databases in 2022: A Year in Review by swyx on Tuesday January 03, 2023 @12:20AM (#34228007)
(just want to say i am extremely impressed by your answers in this thread - very precise and clear)

> there is a lot more opportunity for Neon in serverless, devtools, and edge

serverless i get, but am not sure about the other two. i guess my conception of database business is that it is a volume biz, ie mainly the terabyte-petabyte scale workloads are the valuable enterprise business. high value mixed workloads are a subset of that but i’d imagine “postgres++” would be a good value proposition as well (easy migration with no replatforming).

my impression is that devtools/edge would be relatively low volume. perhaps high margin but not enough to make up for low volume. do i have a misconception here?

Re:Databases in 2022: A Year in Review by nikita on Tuesday January 03, 2023 @12:46AM (#34228177)
So the market looks like this:

- Tier1: Unique large scale workload: think about moving money around. This is ~$10-20Bln market mostly dominated by Oracle, Microsoft and IBM. All next gen scale systems kind of forced to play in this market. Deals are large by sales cycles are long.

- Fleets of tier2 apps. Each enterprise now has a fleet of every major database offering to power lots of apps. To win this market you need to be a top 3 database in the enterprise. Most money is made by SQL Server, but Postgres has a real shot here. This market is driven by developers and SQL Server takes most $$ closely followed by AWS. This is Neon’s bet. That after getting a wedge in a low $$ SMB/hobbyist market Neon takes mid market and then moves to enterprise. The size of this market is also $10-20 Bln.

- OEMs. This is another $5-10 Bln. Neon only cares about dbaas but happily embed in everything cloud.

Everything else is cats and dogs.

Re:Databases in 2022: A Year in Review by swyx on Tuesday January 03, 2023 @03:30AM (#34229198)
great market insight. thank you!
Re:Databases in 2022: A Year in Review by Ataraxy on Monday January 02, 2023 @10:34PM (#34227264)
I actually look forward to Neon but won't actually use it for real until pricing becomes available.
Re:Databases in 2022: A Year in Review by nikita on Monday January 02, 2023 @10:56PM (#34227450)
Stay tuned. It's coming very very soon.
Re:Databases in 2022: A Year in Review by fulafel on Tuesday January 03, 2023 @12:50AM (#34228204)
Regarding separation of storage and compute, do those DBs you mention enable merging the application backend code and db compute node to a single process? Meaning I would deploy my application to the DB compute node and use its runtime, or have the compute node present as a component I can import to my backend code as a library. I think this is one powerful thing about Datomic (and maybe XTDB?).