Re:I Accidentally Saved My Company Half a Million Dollars
by kwillets on Monday October 30, 2023 @09:24PM
(#38078832)
Message queueing triggers a host of psychological needs. Synchronous jobs rarely need monitoring and management features, but move the same work to a queue, and everybody loses their minds.
There's a service that watches the transaction log of your regional replica so that you can make long-poll HTTP requests that return when any change matching your filter is committed.
It makes a lot of sense for a financial risk system, where you end up calculating millions of slight variations on a scenario. The data flow model with aggressive memoization makes this sort of thing much cheaper.
However, I saw plenty of systems written where you'd attempt to write your request to the next key matching some regex (and retry with the next key if it already existed), where your request would contain some parameters and the database key and/or filesystem path where results should be written.