At the bottom of the page on riverqueue.com it appears there's a screenshot of a UI. But I can't seem to find any docs about it. Am I missing something or is it just not available yet?

Looks like it’s underway:

> We're hard at work on more advanced features including a self-hosted web interface. Sign up to get updates on our progress.

The UI isn't quite ready for outside consumption yet but it is being worked on. I would love to hear more about what you'd like to see in it if you want to share.

If you could build a UI similar to Hangire [0] or Laravel Horizon [1], that would be awesome.

[0] https://hangfire.io

[1] https://github.com/laravel/horizon

An Airflow for Gophers?

So excited y'all created this. Through a few job changes I've been exposed to the most popular background job systems in Rails, Python, JS, etc, and have been shocked at how under appreciated their limitations are relative to what you get out of the box with relational systems. Often I see a lot of DIY add-ons to help close the gaps, but its a lot of work and often still missing tons of edge cases and useful functionality. I always felt going the other way, starting w/ a relational db where many of those needs are free, would make more sense for most start-ups, internal tooling, and smaller scale businesses.

Thank you for this work, I look forward to taking it for a (real) test drive!

How do you look at models like temporal.io (service in front of DB) and go-workflows (direct to DB) in comparison? It seems like this is more a step back towards that traditional queue like asynq is, which is where the industry is leaving from to the model of temporal

I don't think these approaches are necessarily mutually exclusive. There are some great things that can be layered on top of the foundation we've built, including workflows. The best part about doing this is that you can maintain full transactionality within a single primary data store and not introduce another 3rd party or external service into your availability equation.

Personally I've found Temporal very limited as a queue: no FIFO ordering (!), no priorities, no routing, etc. It's also very complex when you just want async jobs, and more specialized than a DB or message broker, which can be used for many other things. I think there's a place for both.

How does this compare to https://github.com/vgarvardt/gue?

Not familiar with either project, but it seems gue is a fork of the authors previous project, https://github.com/bgentry/que-go

Yes, there's a note in the readme to that effect although I don't think they bear much resemblance anymore. que-go was an experiment I hacked up on a plane ride 9 years ago and thought was worth sharing. I was never happy with its technical design: holding a transaction for the duration of a job severely limits available use cases and worsens bloat issues. It was also never something I intended to continue developing alongside my other priorities at the time and I should have made that clearer in the project's readme from the start.

Or neoq. https://news.ycombinator.com/item?id=38352778

I don't know why people even use libraries in those languages - assuming you stick to a database engine you understand well then the (main)-database-as-queue pattern is trivial to implement. Any time spent writing code is quickly won back by not having to debug weird edge cases, and sometimes you can highly optimize what you're doing (for example it becomes easy to migrate jobs which are data-dominated to the DB server which can cut processing time by 2-3 orders of magnitude).

It's particularly suited to use cases such background jobs, workflows or other operations which occur within your application and scales well enough for what 99.9999% of us will be doing.

I thought the first rule of queues was never use a database as a queue.

The "problem" is that a database as a queue doesn't infinitely scale, but nothing infinitely scales. If you're starting to reach the limits of Postgres-as-a-queue then you've either done something very wrong or very right to get to that point.

It's probably more correct to say that the engineering effort required to make a Postgres-as-a-queue scale horizontally is a lot more than the engineering effort required to make a dedicated queueing service scale horizontally. The trade-off is that you're typically going to have to start scaling horizontally much sooner with your dedicated queuing service than with a Postgres database.

The argument for Postgres-as-a-queue is that you might be able to get to market much quicker, which can be significantly more important than how well you can scale down the track.

I thought Kafka could relatively infinitely scale

I wouldn't categorise Kafka as a database.

Is there a minimum version of Postgres needed to use this? I’m having trouble finding that information in the docs

Does it do job completion notification?

Along the lines of:

    _, err := river.Execute(context.Background(), j) // Enqueue the job, and wait for completion
    if err != nil {
        log.Fatalf("Unable to execute job: %s", err)
    }
    log.Printf("Job completed")

Does that make sense?

It sounds like you're looking to be able to find out when a job has finished working, no matter which node it was run on. No, River does not have a mechanism for that today. It's definitely something we've talked about though.

Thanks. Yeah, that's definitely a requirement of the queueing systems I'm working with.

It's effectively "run this remote function, and return when you have the result data".

Fan-out RPC (-ish) style stuff, rather than async "queue an email for sending later".

Really cool. I'm working on .Net project that I've also adopted a "single dependency" stance on; that being Postgres. I'm pretty thrilled to see I'm not the only one lol!

I plan to use Orleans to handle a lot of the heavy HA/scale lifting. It can likely stand in for Redis in a lot of cache use cases(in some non-obvious ways), and am anticipating writing a Postgres stream provider for it when the time comes.. Will likely end up writing a Postres job queue as well so will definitely check out River for inspiration.

A lot of postgres drivers, including the .Net defacto Npgsql, support logical decode these days which unlocks a ton of exciting use cases and patterns via log processing.