Re:Why Reactiflux, Vue Land, Yarn, and other open source com...
by shados on Tuesday March 13, 2018 @07:47PM
(#16580965)
Makes sense. I did read that blog before, but I just can't keep thinking about just the cost of storing that much text on disc, even if it was compressed, would quickly add up.
Maybe my mental math is failing me (or I'm overestimating how many messages there are), but that still seemed expensive.
In the end, as long as you don't go out of business, because I'd be very very sad :)
Re:Why Reactiflux, Vue Land, Yarn, and other open source com...
by jhgg on Tuesday March 13, 2018 @08:08PM
(#16581089)
Storage is actually really cheap for this, we don't store the message data in ES, it just builds the inverted indices. I explain this more in-depth on the blog post.
On GCP, 50TB of Persistent SSD is roughly $9000/month before any discounts.
Cost isn't actually too bad, given how we've designed our search schema - and how we lazily index.