> It is not necessary to mark all permanently unavailable resources as "gone" or to keep the mark for any length of time -- that is left to the discretion of the server owner.
On further thought, it seems to me that we necessarily need to keep completed tasks around until asked by a client to delete them. If a task produces an exception, we need to notate that for when the client next asks about the task's status. If the task ran successfully to completion, we need to notate that for when the client next asks. If the client needs to know all this, then it's the client who needs to ask for deletions as well - we shouldn't assume it's OK to remove records about tasks (unless there's a published policy that completed/error data will be purged after a certain period, e.g. 30, 60, 90 days...)
Why doesn't that scale?
> False positive matches are possible, but false negatives are not – in other words, a query returns either "possibly in set" or "definitely not in set". Elements can be added to the set, but not removed (though this can be addressed with a "counting" filter); the more elements that are added to the set, the larger the probability of false positives.
So I don't think they're the right choice for this problem, if you actually depend on the result "not here" vs "never was here" being right.
The only info that needs to be stored is the ID of the finished task. If, say, the ID is expressed as a GUID then we're talking about 4 ints or 2 longs worth of data.
Even so, these records are supposed to be garbage collected.
Also deleting is far from free of side effects.
API client must issue DELETE request, so that server purges it. Until then, server responds with 303 See Other status. Once deleted, 404 Not Found will be returned for subsequent GET /queue/12345 requests. or, garbage collection can be a server’s job to do: once task is complete server can safely remove it and respond with the 410 Gone on subsequent GET /queue/12345 requests.”
How do you distinguish between an id referring to a finished task that no longer exists (410 Gone), and an id that never referred to any task (404 Not Found)? Other than the obvious solution of keeping every task in the DB with a “finished” flag, which doesn’t scale...