News for hackers. Stuff that matters.
Re:Welcome to InfluxDB IOx: InfluxData’s New Storage Engine by _peter_ on Wednesday October 26, 2022 @12:04PM (#33345735)
Isn't InfluxDB rewriting their storage engine for the nth time? It makes me have a little less faith in their project to be honest.
Re:Welcome to InfluxDB IOx: InfluxData’s New Storage Engine by mhall119 on Wednesday October 26, 2022 @12:35PM (#33346137)
The original TSM engine is still used by InfluxDB v2 OSS.

The InfluxDB Cloud platform uses a variation of TSM that's tailored for a distributed SaaS rather than stand-alone nodes (this was originally intended to be used in InfluxDB v2 OSS as well, but alpha-testing showed that the old engine performed better there so it ultimately was reverted for the beta release).

So IOx is really the first major new storage engine in InfluxDB.

Re:Welcome to InfluxDB IOx: InfluxData’s New Storage Engine by c4wrd on Wednesday October 26, 2022 @01:28PM (#33346810)
I would argue the other way and praise them for the storage engine changes. Each iteration has had drawbacks, but based on the real-world reported usage they've made decisions to better support what customers are asking for and actually running into, as opposed to trying to iterate on the same engine over and over and making assumptions of real-world usage. Sure, there are drawbacks, but at the end of the day they're continuing to make good improvements for their customers.
Re:Welcome to InfluxDB IOx: InfluxData’s New Storage Engine by dgnorton on Wednesday October 26, 2022 @01:50PM (#33347047)
Member of the engineering team here - I would break the history into 3 phases:

1) Alpha / Beta phase where we experimented with several off-the-shelf key-value stores (RocksDB, LevelDB, & BoltDB). During this early phase, we learned from observing a wide variety of workloads / use-cases that we needed a custom built engine to achieve our early performance goals. But, using these off-the-shelf key-value stores allowed our (at the time) very small team to focus on developing a useful beta product and gathering user feedback.

2) TSM storage engine for 1.0 - Developed from scratch based on our learnings from phase 1, this was the first production storage engine that shipped with 1.0 in 2016 and carried us through 2.0. It served as the workhorse for 3 - 4 years as both the number of users and size of their workloads skyrocketed, eventually bumping into architectural limits of TSM.

3) IOx - equipped with a larger engineering team and years of experience with a wide variety of workloads and use-cases, IOx was developed to handle rapidly growing time series workloads that users need to handle.

Re:Welcome to InfluxDB IOx: InfluxData’s New Storage Engine by linsomniac on Thursday October 27, 2022 @11:23AM (#33358948)
IOW: The first iteration has to be perfect?

Why that vs. Fred Brooks' "Plan to throw the first one away" idea?