Hope you're doing well BTW!
I didn't expect it to be a perfect system but I don't want to deal with setting up a queuing system right now when there simply isn't enough activity taking place.
Will it fail at some point? Possibly, but it's a great intermediary solution as you state. I for one would be curious to see other solutions to this problem. Are there any resources you'd recommend?
Also, eventual consistency is a weak requirement that, when sufficient, enables a certain level of fault tolerance and low latency not achievable with other consistency levels (see [1]), although other consistency levels may offer good-enough fault-tolerance and latency (rivaling EC in many cases). EC is not in general the solution to the problem of "how do I best distribute my data?"
[1]: http://www.bailis.org/blog/when-does-consistency-require-coo...
Absolutely, but if you can avoid using global locks, which are error prone and reduce reliability, isn't that a good thing?
In other words: if you set out to create Bitcoin in 1998, you might end up running into distributed consensus problems that are difficult enough that you decide to "solve an easier problem"—such that you end up creating Paypal instead.
I think that while this may be true for many requirements, consistency is among the least flexible ones. Besides, losing consistency usually makes your system more complicated, not less. Weak consistency has some real benefits (latency in some cases, tolerance for more failure modes), but simplicity is not usually one of them.
> In other words: if you set out to create Bitcoin in 1998, you might end up running into distributed consensus problems that are difficult enough that you decide to "solve an easier problem"—such that you end up creating Paypal instead.
I don't think that the design problems behind Bitcoin are any easier than distributed consensus, which, BTW, is not necessary for quite a few consistency levels.
Banks have generally solved this by batching updates, but yes, it could be unsettling for the customer in a similar situation. There are ways around that though, for example you update a cache that stores the data instantly and then let the eventually consistent source of truth update the cache again if it is not in sync, or perhaps have a short timeout on the cache, so the customer gets a consistent and smooth experience but the source of truth is still accurate, eventually.
Leader election generally[1] requires consensus among distributed processes, and global locks generally[1] require consensus as well. The benefit of this is that both problems can be solved on top of a common consensus implementation which is what http://atomix.io does.
1: I say generally because you can do fancy things with fancy clocks to avoid running operations through a quorum under certain circumstances, but these carry caveats that preclude them from being reliable enough to use in many use cases.
For example, a lot of people say, "updating a bank balance". But even that can easily be eventually consistent. You send increments and decrements and settle up at the end. Yes, your balance might go negative, but that's what ATM limits are for -- the most you could go negative is that limit. Most other systems can be written the same way. And for non-numeric values, there are always vector clocks or timestamps or quorum to determine which update came next, or you could send a client a list of all recent updates and let the client "do the right thing".