News for hackers. Stuff that matters.
Re:Cloud Memorystore: A fully managed in-memory data store s... by panarky on Wednesday May 09, 2018 @01:59PM (#17032132)
Their post isn't very clear on persistence, here's how I read it.

1) Yes, they're using Redis persistence (RDB, AOF, or a Google-written persistence layer, it's not specified).

2) Basic Tier scales by tearing down the smaller Redis instance and migrating the data (???) to a larger instance. This causes downtime, but not lost data.

3) Standard Tier migrates using replication during scaling, so there's little downtime but if data is in flight at cutover, it could be lost.

4) And "coming soon" you can migrate data in and out of Cloud Memorystore by importing and exporting RDB files.

Re:Cloud Memorystore: A fully managed in-memory data store s... by gopalashok on Wednesday May 09, 2018 @02:29PM (#17032463)
Full disclosure, I'm the product manager for Cloud Memorystore

1) We are not using persistence at all in beta, this is something we are working towards

2) Scaling of basic tier will result in a full cache flush and will cause downtime during the scaling process.

3) That's correct

4) Yes we will be making Import-Export of RDB very soon

Re:Cloud Memorystore: A fully managed in-memory data store s... by sitkack on Wednesday May 09, 2018 @03:25PM (#17033045)
> 1) We are not using persistence at all in beta, this is something we are working towards

Persistence is a key feature of Redis, it isn't just a fancy memcached. This severely limits the applicability of this offering.

Re:Cloud Memorystore: A fully managed in-memory data store s... by jkaplowitz on Wednesday May 09, 2018 @05:27PM (#17034269)
So does the beta label. :) Both will change in time, from what the PM said.
Re:Cloud Memorystore: A fully managed in-memory data store s... by dannyw on Wednesday May 09, 2018 @09:36PM (#17035829)
It’s like releasing a beta for DNS that doesn’t support IPv6 or CNAMES.
Re:Cloud Memorystore: A fully managed in-memory data store s... by rkangel on Thursday May 10, 2018 @05:00AM (#17037284)
Exactly. It will be still be useful to some people, it allows them to get something working end-to-end, and then iterate on it to provide the features for more people.
Re:Cloud Memorystore: A fully managed in-memory data store s... by sitkack on Thursday May 10, 2018 @09:22PM (#17044324)
Not providing persistence is setting people up to lose data. It is the wrong move.
Re:Cloud Memorystore: A fully managed in-memory data store s... by russell_h on Wednesday May 09, 2018 @04:51PM (#17033965)
This is a great looking product, thanks for following up here.

Can you speak to what kind of durability you expect to achieve once persistence is in place? Is it reasonable to treat this as a durable datastore and simply have a disaster recovery plan in place? Or should applications be prepared to handle occasional minor data loss?

Re:Cloud Memorystore: A fully managed in-memory data store s... by hamandcheese on Wednesday May 09, 2018 @04:37PM (#17033817)
> We are not using persistence at all in beta

Maybe don’t advertise it as a “data store” then.

Re:Cloud Memorystore: A fully managed in-memory data store s... by jkaplowitz on Wednesday May 09, 2018 @05:26PM (#17034261)
Transient caches are still a useful type of data store for better performance... That said, they're noting it as a goal and a beta limitation, so they're not satisfied with pure transience either.
Re:Cloud Memorystore: A fully managed in-memory data store s... by hamandcheese on Wednesday May 09, 2018 @11:09PM (#17036119)
That’s fine, but it should be made way more obvious what this is and what it isn’t. This goes against expectations.
Re:Cloud Memorystore: A fully managed in-memory data store s... by antonvs on Thursday May 10, 2018 @02:42AM (#17036866)
Its name is Cloud MemoryStore, not Cloud Diskstore. That should set expectations right there.
Re:Cloud Memorystore: A fully managed in-memory data store s... by hamandcheese on Thursday May 10, 2018 @03:03AM (#17036946)
They use the term “data store” multiple times, not to mention:

> and features like persistence, replication and pub-sub.

(Customer testimonial) > We have used Redis on everything from storing asynchronous task queues for tens of thousands of CPUs to a centralized persisted key-value pair store for the feature vectors output by our ML models

Additionally, “persistence” isn’t mentioned at all in the “Coming soon to Cloud Memorystore” section.

I’m not trying to be a pedant, I just think it’s dangerous that it’s not made abundantly clear that there is no persistence.

Re:Cloud Memorystore: A fully managed in-memory data store s... by dudus on Wednesday May 09, 2018 @02:04PM (#17032198)
Why do you think they are using persistence at all, and why do you think Basic Tier doesn't have data loss on scaling?

I read it differently. When they say full cache flush doesn't that imply full database flush, I was thinking all data gets wiped on scaling operation.

Re:Cloud Memorystore: A fully managed in-memory data store s... by nodesocket on Wednesday May 09, 2018 @02:12PM (#17032292)
Agree I don't think it is persistent. "a full cache flush" represents all data loss to me.
Re:Cloud Memorystore: A fully managed in-memory data store s... by panarky on Wednesday May 09, 2018 @02:28PM (#17032447)
Could be I read this one wrong.

There's a Redis command called FLUSHALL that deletes all the data.

https://redis.io/commands/flushall

Re:Cloud Memorystore: A fully managed in-memory data store s... by dudus on Wednesday May 09, 2018 @02:18PM (#17032351)
Wouldn't that also imply full data loss on server restarts?