News for hackers. Stuff that matters.
Re:Cloud Filestore, high-performance file storage for GCP users by _wmd on Tuesday June 26, 2018 @12:37PM (#17402044)
They've undercut Amazon EFS by 33% for the budget option ($.20/gb vs. $.30/gb). Hope this kicks off a little pricing war - shared filesystem is a hugely useful solution for migrating old apps, would be nice if it were cheap. Both options are still grossly overpriced (IMHO)
Re:Cloud Filestore, high-performance file storage for GCP users by panarky on Tuesday June 26, 2018 @01:51PM (#17402669)
Amazon EFS bills for usage while Google bills for provisioned capacity.

So unless you run GCP at a very high % of provisioned capacity, it's more expensive.

I'm pretty confident EFS can't match GCP's 700 MB/s and 30,000 IOPS though.

Re:Cloud Filestore, high-performance file storage for GCP users by jkaplowitz on Tuesday June 26, 2018 @01:58PM (#17402720)
Resizing on demand without downtime up to a maximum of 64TB will allow running at that very high % of provisioned capacity.
Re:Cloud Filestore, high-performance file storage for GCP users by halbritt on Tuesday June 26, 2018 @03:42PM (#17403451)
I did a fair number of tests last year and couldn't get anything like that. If anyone is interested, the poorly formatted output of that is here:

https://github.com/halbritt/benchmarks

Re:Cloud Filestore, high-performance file storage for GCP users by tjenkinsqs on Tuesday June 26, 2018 @08:08PM (#17404901)
Can you please elaborate a little? EFS scales with size, i.e. did you have at least a few hundred GB on there or was it mostly empty? Roughly how many IOPS did you get?
Re:Cloud Filestore, high-performance file storage for GCP users by halbritt on Wednesday June 27, 2018 @10:25PM (#17413602)
IIRC, I populated the thing with 10TB or thereabouts.

For small block sizes, I got hundreds of IOPS.

Latency was pretty terrible in almost all cases.

I wrote a great deal on this topic for internal consumption. Those benchmarks weren't really meant for the general public. My biggest conclusion is that EFS isn't useful for any workload where performance is a concern. Unfortunately, it's priced so high, that I'd never consider using it otherwise.

Re:Cloud Filestore, high-performance file storage for GCP users by mankash666 on Wednesday June 27, 2018 @02:22AM (#17406136)
You can always dynamically resize at the rate of 1GB.

That's still a drawback as your app needs to manage dynamic resizing. While aws may not be more expensive, it's more convenient

Re:Cloud Filestore, high-performance file storage for GCP users by chucky_z on Tuesday June 26, 2018 @02:09PM (#17402794)
We use ~100GB with a lot of in/out for EFS, as a kind of 'nearline S3,' mounted to a hundred ec2 instances or so. It replaced an NFS server doing the same thing, and it ended up being slightly cheaper cause you don't pay for the compute, only the storage (efs vs. ebs + ec2).

Also I haven't had a single issue with performance. It's not particularly busy though (~800 iops?).

Re:Cloud Filestore, high-performance file storage for GCP users by toomuchtodo on Tuesday June 26, 2018 @04:09PM (#17403613)
We’ve seen horrendous performance on EFS unless you’ve provisioned quite a bit of storage on the volume.
Re:Cloud Filestore, high-performance file storage for GCP users by X-Istence on Tuesday June 26, 2018 @05:43PM (#17404230)
Even if you have quite a bit of storage. ~40TB on EFS, and yet performance is abysmal due to the access time being about ~3ms.

So large files work well, but if you have many smaller files it's killer.

We built our app to do caching locally into NVMe drives though, so we async pull data from EFS and push back as necessary.

Definitely going to give Filestore a spin and see if it is a good fit for our use cases.

Re:Cloud Filestore, high-performance file storage for GCP users by stonewhite on Tuesday June 26, 2018 @04:40PM (#17403819)
Underlying S3 bits do bite you when you deal with lots of small files. Our team even considered maintaining NFS or Gluster servers over horrible EFS performance.
Re:Cloud Filestore, high-performance file storage for GCP users by objectivefs on Tuesday June 26, 2018 @05:49PM (#17404269)
Another file system that works well with lots of small files and that doesn't require maintaining any extra servers is ObjectiveFS[0]. It is a FUSE based log structured file system (with snapshots) using GCS or S3 for log storage with memory cache and (optional) disk cache for performance.

[0] https://objectivefs.com

Re:Cloud Filestore, high-performance file storage for GCP users by mtgx on Tuesday June 26, 2018 @02:28PM (#17402942)
As Amazon starts winning fat government contracts for unaccountable sums of money, it will probably care less and less about offering "value" to its smallest private sector customers.
Re:Cloud Filestore, high-performance file storage for GCP users by privacypoller on Tuesday June 26, 2018 @03:00PM (#17403179)
Amazon had better improve its customer service offering, or ramp up its partner network if it wants to win big "enterprisy" clients like government then. The HN crowd may look at the Oracles and IBMs as super-expensive dinosaurs, but they do offer an incredibly high level of help and support.
Re:Cloud Filestore, high-performance file storage for GCP users by pnathan on Tuesday June 26, 2018 @06:34PM (#17404495)
AWS has excellent customer service offerings after a certain price point. I would check the "Business" and "Enterprise" plans for examples of how to get the good support.

You get an account manager rep and upon occasion, interactions with AWS engineers, depending on what you're doing and how exciting it is.

One might say a lot about AWS - but I don't fault them on the customer service front for a serious business spending serious money.

Re:Cloud Filestore, high-performance file storage for GCP users by zwily on Tuesday June 26, 2018 @06:07PM (#17404349)
Amazon already has great customer service for big clients, and they already have lots of government clients too. (Including a completely private region for the CIA and other intelligence services.)
Re:Cloud Filestore, high-performance file storage for GCP users by Yeroc on Tuesday June 26, 2018 @03:19PM (#17403305)
The same applies to Google. Microsoft appears to have the leg up on Oracle & IBM as far as winning "enterprisy" clients.
Re:Cloud Filestore, high-performance file storage for GCP users by gigatexal on Tuesday June 26, 2018 @03:43PM (#17403463)
Ever try getting a SQL support request to an engineer actually at Microsoft and not one outsourced and get a response within weeks? Yeah, not easy. Or if you get a response it's the equivalent of "have you tried turning it off and then on again?".
Re:Cloud Filestore, high-performance file storage for GCP users by manigandham on Tuesday June 26, 2018 @05:25PM (#17404128)
Yes, we had a SQL support case replied to by email, and then followed-up via phone/screenshare to have it resolved that day.

The only constant with tech support is that it's highly variable and depends on many factors. Enterprise MSDN/SQL support is not bad though and you should've gotten a quick reply if you're using the proper channels with an appropriate support plan.

Re:Cloud Filestore, high-performance file storage for GCP users by gigatexal on Tuesday June 26, 2018 @05:30PM (#17404155)
The shop I came from has been a Microsoft and VMware shop for years and we spend millions a year on MSDN licensing. The support has been kinda shitty. But I’m not there anymore. I do remember in DBA meetings laughing about how crappy it is. And why do we pay so much for offshore support techs when they should be local on premises engineers.
Re:Cloud Filestore, high-performance file storage for GCP users by qaq on Tuesday June 26, 2018 @03:46PM (#17403480)
Amazon does too when you are well into few mil. per month range they are pretty responsive (used to work for smaller shop 50k/month range spend AWS) the difference is night and day.
Re:Cloud Filestore, high-performance file storage for GCP users by oh-kumudo on Tuesday June 26, 2018 @02:40PM (#17403028)
Are they? I think Startup customers are very valuable since they are most cost-effective thus providing very good feedbacks.
Re:Cloud Filestore, high-performance file storage for GCP users by paulddraper on Tuesday June 26, 2018 @05:23PM (#17404115)
EFS is very slow. It doesn't work with AWS Fargate or Lambda.

I'm sure there is some good use for it; I just haven't found one yet.