Re:Joy and Pain of Using Google BigTable
by jonomacd on Friday January 25, 2019 @06:52AM
(#18997235)
Interestingly, for us scaling up doesn't really solve the short term unavailability. It seems to be only somewhat related to load as it does seem to hit more often at high traffic times but we have also seen it at low traffic times.
Putting in that cache is a great move. Cache is challenging for us as we get hits over a very wide range of keys.
The Key Visualizer has been a huge help but there's still not enough metrics and tooling to understand when things do go wrong or what is happening behind the scenes. Luckily we have a cache sitting in front of Bigtable for reads that allows us to absorb most of the described intermittent issues because cost has prevented us from doing any sort of replication.