KraftyOne

This is something we mention in the third section of the article: dead tuples and autovacuum caused real performance hits, which we (partially) mitigated through index optimization (minimizing the number and size of indexes, and making indexes partial).

PgQue is a really interesting system! However, its semantics are quite a bit more similar to Kafka than to a job queue, which is good for some workloads and not for others. For example a truncation-based deletion system is fast but inflexible, and not suitable for a job queue system because a single long-running job (and DBOS supports workflows that run for months) can block truncation.

The core optimization is to buffer notifications in-memory and send them in a batch instead of sending them as part of every transaction. So that's a general-purpose optimization for Postgres apps using LISTEN/NOTIFY.

If you mean the optimizations coming in Postgres 19, the original post addresses this:

> As an aside, there’s been some online discussion of a Postgres patch (https://github.com/postgres/postgres/commit/282b1cde9dedf456...) related to this issue. This patch (to be released in Postgres 19) does not remove the global lock or fix the bottleneck we observed. Instead, it optimizes the narrower case where there are many notification channels and each listener is waiting only on a specific channel.

To be clear, it's not a custom patch to pg itself, but an application-side buffering and batching optimization.

Outbox's power is that it turns an atomicity problem into an idempotency problem. You atomically write to the outbox, then you have an idempotent "workflow" that processes events from the outbox. This turns "at most once" semantics (where an event could be dropped entirely) to "at least once" semantics (where the event processing could run multiple times). For many systems, that's a big improvement.

You still need idempotency for side effects outside your database, that's true (and fundamental to durability). But now you get exactly-once semantics for operations on your database, which can be quite valuable if your workflow performs many such updates or they're particularly complicated or stateful.

That's what the post is about! Once you're doing that, you really do have transactions between the state and the queue.

Every item will be written to the queue exactly once (as the update is transactional). Queue processing may need at-least-once semantics, yes, depending on what exactly you're doing.

You build a distributed system on top of this! For example, you may have many distributed workers durably executing workflows from the Postgres-backed task queue. The Postgres transactions allow you to atomically perform operations spanning both your task queue and your business data.

Here's another blog post about how a Postgres-backed task queue can run at scale: https://www.dbos.dev/blog/making-postgres-queues-scale

Yes, the core design is building a workflow system on a database--essentially, replacing the central orchestrator most workflow systems use with a Postgres database. This previous blog post goes into more detail: https://www.dbos.dev/blog/postgres-is-all-you-need-for-durab... (HN discussion: https://news.ycombinator.com/item?id=48313530)

The key is that the UDF's enqueue is transactional with the database update. Let's say the database update is inserting a new order. This provides the guarantee that if a new order is inserted, a job to process the order is also enqueued. It's impossible for a new order to be inserted without its processing job also being enqueued. Then the durable workflow/queue system is responsible for making sure the processing job, once enqueued, actually executes.

Exactly! It's a distributed system, with many processes performing work in parallel, with a central database as a coordination point, used as little as possible. A mutex wouldn't get quite the same performance :)

Unfortunately, DSQL is still lacking many core Postgres features (most notably foreign keys) that DBOS and DBOSify depend on. DBOS works with most flavors of Postgres though, including regular Aurora/RDS, Cloud SQL, AlloyDB, Supabase, Neon, CockroachDB, and others.

Take any Temporal example and swap the imports and connection strings, it's a drop-in replacement :)

We'd need DBOS in C# first! It's something we're considering for the future--if you're working in C#, happy to chat about it, reach out at [email protected].

Exactly, Postgres as an orchestrator instead of a Temporal server as an orchestrator. This page goes into more detail (for DBOS, but DBOSify is the same principle but using the Temporal API directly): https://docs.dbos.dev/explanations/comparing-temporal