News for hackers. Stuff that matters.
Re:Rust is the future, C is the new Assembly: Josh Triplett ... by jandrewrogers on Wednesday August 28, 2019 @09:41PM (#20825915)
For my use cases we can ignore the case of self-referential structs. Slapping "unsafe" on everything isn't equivalent to C++ because Rust is less expressive in some important ways.

I mostly work on high-end database engines, which appear to violate assumptions of Rust about what a sensible memory model can look like. In a modern database engine, almost all of your working memory is unavoidably paged, opaque, and directly DMA-ed. The concept of a lifetime cannot be attached to a memory address and virtually all of your data structures are "unsafe" in a Rust sense. You have to consider implications like destructors being effectively non-deterministic.

Fortunately, schedule-based safety models are transparent to developers, do not require "borrowing" so it works well with references not visible at compile-time, and all references can be mutable. To be clear, these models don't generalize to all software either, they are just well-suited to the requirements of high-throughput server engines. High-scale database engines are effectively single-threaded, so the only cost is a relatively trivial number of non-atomic ref counts. The underlying C++ to make this be handled automagically from a developer perspective is not trivial though.

Rust's memory safety model looks like it was designed for more traditional applications, like web browsers. Modern server software that is I/O intensive is going to look pretty similar to the above if performance matters, and that is a rough fit for Rust's memory model.

Re:Rust is the future, C is the new Assembly: Josh Triplett ... by steveklabnik on Wednesday August 28, 2019 @09:46PM (#20825939)
Have you looked at TiKV/TiDB? I’m curious what you think of them.
Re:Rust is the future, C is the new Assembly: Josh Triplett ... by jandrewrogers on Thursday August 29, 2019 @02:02PM (#20832024)
TiDB is built as a set of abstractions on top of RocksDB, inheriting weaknesses intrinsic to that style of architecture. Two things I would call out about the design:

First, loose coupling through the layers of the stack probably gives up an order of magnitude in throughput on the same hardware versus a state-of-the-art tightly coupled architecture for many workloads. Many optimizations are enabled when high level orchestration and operation semantics (like a SQL join) are understood at every level of the stack down to DMA scheduling. Most open source databases use loose coupling because (1) it is much simpler to design and (2) developers can reuse third-party code for parts they lack the expertise or resources to implement themselves.

Second, RocksDB is a modern implementation of an obsolete design. As a practical matter, this has a high cost in terms of storage scalability and throughput. Many open source projects use it because they don't have a clear idea of how to design something better. Applications where storage engine performance and scalability are critical, e.g. real-time sensor data models, is an area where open source is not remotely competitive at the moment.

Unfortunately, these criticisms could be leveled at most open source database engine designs, which tend to optimize for expediency and ease of implementation rather than performance, and with little awareness of how much performance is being sacrificed. The divergence between academic literature and state-of-the-art in databases is now almost unrecognizable, and the handful of people with expertise in high-end implementation are not working on open source systems.

Re:Rust is the future, C is the new Assembly: Josh Triplett ... by infinite8s on Friday August 30, 2019 @07:26AM (#20837552)
How are schedule-based safety models different than arena-based models?