mfreed

- C is much better integrated into Postgres, doesn't require pgrx. Given also experience developing pg_textsearch, and the acceleration from AI-assisted development and testing, have moved back to more C with extensions.

- Two reasons. First, decision to move back to C. Second, if there ultimately isn't much code overlap, better to keep them separate for easier adoption (smaller footprint and security/correctness envelop)

A lot of great questions, feedback, and already some PR contributions today about pg_vectorsearch. I've written a short technical deep dive into the architecture of its PRISM index to answer some, thought I'd share:

https://www.linkedin.com/pulse/inside-pgvectorsearch-archite...

But one of the great things about open source and PostgreSQL's extension architecture is that these projects can coexist! pg_vectorsearch can work directly with pgvector's existing data types, so developers can use either index.

This isn’t the first vector index we’ve built and open-sourced for PostgreSQL. In 2024, Tiger Data released pgvectorscale, which used a graph-based indexing algorithm inspired by Microsoft’s DiskANN research. We deliberately chose a disk-optimized design so that indexes could scale beyond available RAM. Our initial benchmarks were promising, but as adoption grew and we tested against a wider variety of datasets, particularly low-dimensional ones, we encountered a troubling problem: portions of the graph could become unreachable, causing recall to degrade. We searched the literature for a principled solution but couldn’t find one, and the heuristics we tried worked on some datasets but not others.

That’s a frustrating failure mode for a search index. When queries are slow, you can measure the problem and tune accordingly. But when recall degrades, you might not even notice until users start questioning the quality of their search results. And telling them to experiment with yet another set of tuning parameters isn’t the experience we want to provide.

Meanwhile, partitioned-list approaches reminiscent of IVFFlat were getting a second life, as advances in compression and optimization addressed many of their old limitations. More importantly, they offered stronger guarantees than we could achieve by patching a graph-based index. We came to believe they were a better foundation, and decided to start over.

With the introduction of pg_vectorsearch, we are shifting the majority of our vector-search development efforts to this new architecture. The existing pgvectorscale extension, which is stable and handling production workloads currently, will enter maintenance mode.

Yes, TigerFS works well with local PG - in fact, all of my testing framework uses this setup.

Big upside of this is latency; main downside is that such a deployment is primarily "single user mode."

One very interesting approach of this is ability to transparently share state / collaborate across machines, with these transactionally serialized at the common database.

I primarily work with Postgres, including both at scale and in very ephemeral environments. So just haven't looked at all what doing with this SQLite would mean.

The 5ms write latency is because the backend distributed block storage layer is doing synchronous replication to multiple servers for high availability and durability before ack'ing a write. (And this path has not yet been super-performance-optimized for latency, to be honest.)

We're continuing to evaluate demand for multi-region clusters, love to hear from you.

The docs do say, however, "If the volume has been impaired for more than 20 minutes, you can contact the AWS Support Center." [0] which suggests its some expected cleanup/remount interval.

That is, it is something that we regularly encounter when EC2 instances fail, so we were sharing from personal experience.

[0] https://docs.aws.amazon.com/ebs/latest/userguide/work_volume...

Our existing Postgres fleet, which uses EBS for storage, still serves thousands of customers today; nothing has changed there.

What’s new is Fluid Storage, our disaggregated storage layer that currently powers the new free tier (while in beta). In this architecture, the compute nodes running Postgres still access block storage over the network. But instead of that being AWS EBS, it’s our own distributed storage system.

From a hardware standpoint, the servers that make up the Fluid Storage layer are standard EC2 instances with fast local disks.

- EBS typically operates in the millisecond range. AWS' own documentation suggests "several milliseconds"; our own experience with EBS is 1-2 ms. Reads/writes to local disk alone are certainly faster, but it's more meaningful to compare this against other forms of network-attached storage.

- If durability matters, async replication isn't really the right baseline for local disk setups. Most production deployments of Postgres/databases rely on synchronous replication -- or "semi-sync," which still waits for at least one or a subset of acknowledgments before committing -- which in the cloud lands you in the single-digit millisecond range for writes again.

That's different from a data warehouse or many traditional "OLAP" use cases, where you might dump a big dataset statically, and then people will occasionally do ad-hoc queries against it. This is the big weather dataset file sitting on your desktop that you occasionally query while on holidays.

So it's less about "can you store weather data", but what does that use case look like? How are the queries shaped? Are you saving a single dataset for ad-hoc queries across the entire dataset, or continuously streaming in new data, and aging out or de-prioritizing old data?

In most of the products we serve, customers are often interested in recent data in a very granular format ("shallow and wide"), or longer historical queries along a well defined axis ("deep and narrow").

For example, this is where the benefits of TimescaleDB's segmented columnar compression emerges. It optimizes for those queries which are very common in your application, e.g., an IoT application that groups by or selected by deviceID, crypto/fintech analysis based on the ticker symbol, product analytics based on tenantID, etc.

If you look at Clickbench, what most of the queries say are: Scan ALL the data in your database, and GROUP BY one of the 100 columns in the web analytics logs.

- https://github.com/ClickHouse/ClickBench/blob/main/clickhous...

There are almost no time-predicates in the benchmark that Clickhouse created, but perhaps that is not surprising given it was designed for ad-hoc weblog analytics at Yandex.

So yes, Timescale serves many products today that use weather data, but has made different choices than Clickhouse (or things like DuckDB, pg_analytics, etc) to serve those more operational use cases.

- https://www.timescale.com/blog/what-is-clickhouse-how-does-i...

Sidenote: Timescale _does_ provide columnar storage. I don't believe that the blog author focused on this as part of insert benchmarks:

- Timescale columnar storage: https://www.timescale.com/blog/building-columnar-compression...

- Timescale query vectorization: https://www.timescale.com/blog/teaching-postgres-new-tricks-...

https://www.timescale.com/blog/what-is-clickhouse-how-does-i...

(Timescaler)

https://www.timescale.com/blog/building-columnar-compression...

Staring at a >trillion rows in a TimescaleDB hypertable on PostgreSQL.

https://www.timescale.com/blog/expanding-the-boundaries-of-p...

The most common approach here is just to store the step of "dynamic" labels in JSON, which can be evolved arbitrarily.

And we've found that this type of data actually compresses quite well in practice.

Also regarding compression, Timescale supports transparent mutability on compressed data, so you can directly INSERT/UPDATE/UPSERT/DELETE into compressed data. Under the covers, it's doing smart optimizations to manage how it asynchronously maps individual mutations into segment level operations to decompress/recompress.

(Timescale cofounder)

mike (at) timescale or DM on twitter?

Our experience is that isn't a common use case for Athena, while it is the primary use case for TimescaleDB.

Even without this new capability of S3-based storage, Timescale's native columnar compression often gets like 95% storage reduction, even while staying fully in the PostgreSQL ecosystem.

(We often see our users amazed about how much less storage/disk they require after migrating from RDS.)

https://docs.timescale.com/timescaledb/latest/how-to-guides/...

So I agree that it's good for cold storage, but it's a bit nuanced. For example, you rarely see small random queries to old historical data, but you do often see larger scans over historical data. And in those cases, the throughput you get from S3 is actually quite high (especially that we've engineered with with proper columnar compression and row group/columnar exclusion). Which is very different from a latency bounded workload where you have a lot of small random reads, which is much more common in CRUD-like workloads.

Also, with Timescale, you have the ability to build continuous aggregates (incrementally materialized views). So you can have the raw data (or even lower levels of rollups) that get tiered into S3, while the more frequently accessed rollups can remain in hot storage.

(Timescale co-founder)

Yes you can. (Timescale co-founder here)

The vast majority of people using TimescaleDB use our community/open-source software (rather than managed Cloud), and vast vast majority of those use the TSL (Community) edition, including many as part of their SaaS service.

It's a three-part test for asking "is this a value added service?" [0] Given the way your have described your service, sounds like a clear "Yes".

- Is your SaaS service primarily different than a database product/service? Yes.

- Is the main value of your SaaS service different than that of a time-series database, and you aren't primarily offering your SaaS service as a time-series database? Yes.

- Are users prevented from directly defining internal table structures through the database DDL? Presumably yes.

[0] https://www.timescale.com/legal/licenses#section-3-10-value-...

- Gorilla compression for floats

- Delta-of-delta + Simple-8b with run-length encoding compression for timestamps and other integer-like types

- Whole-row dictionary compression for columns with a few repeating values (+ LZ compression on top)

- LZ-based array compression for all other types

https://www.timescale.com/blog/building-columnar-compression...

So as to your question, just turn on compression; it's very common to see 94-97% reduction in storage.

(Timescale co-founder)

More concrete, we see a lot of web3/crypto use cases, and making a wallet ID, NFT name, or ticker as a top-level considerations.

E.g., use your contract address as the segmentby field for compression.

The Timescale License was introduced in late 2018, although we never _relicensed_ any of our Apache 2 code, we only created a space for _new_ capabilities to be developed under these terms.

There were a few changes to the license in Sept 2020, but those only liberalized a few clauses (including from feedback we received from the Hacker News community, including the "right to repair" and "right to improve"). At the time, we also moved any remaining "enterprise" features from an Enterprise/Paid license into the TSL/Community/Free license.

https://www.timescale.com/blog/building-open-source-business...

We haven't changed anything since, nor have we ever moved any Apache-2 code into the TSL.

(Timescale cofounder)

It allows us to cohort performance data across data stored in others microservice databases (e.g., by account types, projects, billing data, etc.). JOINs across foreign data wrappers using TimescaleDB + Postgres, all within the database and without ETL or application code needed.

So you could look at Prometheus data for your trialers vs. customers, for customers running more than X services, for customers that pay more than $X per month or have been a customer for more than 6 months, etc.

It's super useful across operations, support, product, customer care, and more...

The link above is the right slack group, but you need to go in through the URL I provided to get the right invite tokens. Sorry, Slack isn't fully designed as a community platform, even though many use it for such.