News for hackers. Stuff that matters.
Re:Cloud Spanner is now production-ready by gfodor on Tuesday May 16, 2017 @12:56PM (#14351125)
So, can someone explain why this isn't the RDBMS holy grail? (serious question)
Re:Cloud Spanner is now production-ready by brandur on Tuesday May 16, 2017 @01:57PM (#14351737)
I don't think there's any question at this point that Spanner is really cool. The vast majority of cloud databases out there (even brand-new ones like Azure's Cosmos) are still implicitly trying to convince you that not having ACID transactions is something that won't matter very much, and that scale is by far more important, which isn't true. Google hasn't taken that approach with Spanner, and it's a breath of fresh air.

I'd throw one downside out there though: lock in. While queries use standard SQL syntax [1], all write operations are performed via a very non-standard GRPC API [2]. I think the team certainly considered this and weighed the trade offs, but it does mean that after you're on Spanner, you're on Spanner forever.

[1] https://cloud.google.com/spanner/docs/query-syntax

[2] https://cloud.google.com/spanner/docs/reference/rpc/

Re:Cloud Spanner is now production-ready by boulos on Tuesday May 16, 2017 @02:01PM (#14351779)
We're hoping to add more compatibility layers, but the semantics of reading updates within a transaction are what gave the team pause. If you don't need that semantic, it's relatively straightforward (I assume we'll see this happen either directly by us, or from the community).

Disclosure: I work on Google Cloud, but not on Spanner.

Re:Cloud Spanner is now production-ready by fdsfsafasfdas on Tuesday May 16, 2017 @03:27PM (#14352483)
Any chance that Google Cloud will ever expose access to F1?
Re:Cloud Spanner is now production-ready by boulos on Tuesday May 16, 2017 @04:09PM (#14352859)
The point of the SQL layer now merged into Spanner is so that people don't have to run F1 or something like it on top. I'd expect continual improvements there, especially if enough folks say "I'll move if X"
Re:Cloud Spanner is now production-ready by fdsfsafasfdas on Tuesday May 16, 2017 @04:16PM (#14352916)
Well, the hope would be that it would allow read-write-read sql transactions, which to my understanding will never be implemented in Spanner.
Re:Cloud Spanner is now production-ready by bogomipz on Tuesday May 16, 2017 @06:48PM (#14354001)
Any insight into why read-write-read sql transactions will never make it into Spanner?
Re:Cloud Spanner is now production-ready by elvinyung on Tuesday May 16, 2017 @10:34PM (#14355183)
Not sure if "never", but from the new Spanner paper[1]:

> While we have seen comparatively little demand for this feature internally, supporting such semantics improves compatibility with other SQL systems and their ecosystems and is on our long-term radar.

[1] http://dl.acm.org/authorize?N37621

Re:Cloud Spanner is now production-ready by bogomipz on Wednesday May 17, 2017 @10:51AM (#14358944)
Thanks for the clarification and the link.
Re:Cloud Spanner is now production-ready by curiousDog on Tuesday May 16, 2017 @02:10PM (#14351867)
Also important to note that it's a NewSQL system, not a full blown RDBMS. You can't lift and shift your MySQL application and expect full blown horizontal scalability and such.
Re:Cloud Spanner is now production-ready by makmanalp on Tuesday May 16, 2017 @02:48PM (#14352194)
I'm surprised no one has challenged this yet - if there is anything that cloud spanner does better than most things out there, it's "full-blown horizontal scalability"! There are very few systems out there that have tackled the issue of scaling beyond cross-AZ in a non-bullshit way.
Re:Cloud Spanner is now production-ready by dsp1234 on Tuesday May 16, 2017 @03:32PM (#14352524)
You can't lift and shift your MySQL

I'm surprised no one has challenged this yet

Are you saying that a company could lift and shift MySQL then?

Re:Cloud Spanner is now production-ready by _rk8k on Tuesday May 16, 2017 @02:11PM (#14351878)
For me, this immediately discounts it:

> $0.90 per node per hr

That's $648/mo in pure overhead.

I'm not saying there should be no overhead but it shouldn't be that high.

They also optimistically encourage you to try it with a "free" trial. The "free" trial, assuming you haven't used GCP before, gives you $300 credit, which is enough for less than 2 weeks of a single Spanner node.

Plus according to https://cloud.google.com/spanner/docs/instance-configuration, they recommend a minimum of 3 nodes, or $1800/mo.

I know this is targeted at businesses but it'd be nice if we could get smaller "nodes" or something to lower the barrier to entry. You shouldn't have to go straight from nothing to 10k QPS.

Re:Cloud Spanner is now production-ready by boulos on Tuesday May 16, 2017 @02:21PM (#14351980)
In case people don't do the math, you (seem to be) are suggesting that Spanner should be $9/month. I certainly argued for a "shared Spanner" mode, that wouldn't give you a dedicated setup. The benefit is that the minimum "install" would be super low, allowing you to start on a slice and move upwards (like many teams at Google!).

The reality is that $8k/year isn't that much for a company. You can all share it, get quite a bit out even the smallest deployment and so on. However, compared to the work to slim down the minimum deployment for Spanner, this was honestly a reasonable outcome (that is, I certainly wouldn't have wanted to see the team delay a year to build a shared multitenancy version).

Additionally, Spanner does things for you that MySQL et al. don't. Having an automagic Regional (and eventually Global if you'd like) database without dealing with sharding is worth $8k/year even to me. So even if it could fit on $10/month of hardware, I don't begrudge them for charging a service fee, rather than saying "This is how much cores, RAM, disk and flash this eats".

Disclosure: I work on Google Cloud and obviously have a vested interest in you paying us :).

Re:Cloud Spanner is now production-ready by Veratyr on Tuesday May 16, 2017 @04:22PM (#14352971)
I don't know where the $9/month number comes from but as someone else said, I edited my post (added a bunch of things) so maybe I edited out something that could be read that way.

Basically yes, I'd like a shared Spanner. If $648/mo gets you 10k read/2k write, then for ~$9/mo you should be able to get roughly 140 qps read and 9 qps write, so long as cost scales roughly linearly. Assuming you can spread this over the month (1000qps here, 0qps there), this would be super appealing to a lot of smaller projects I think.

I think the way it is, it's exclusively a business tool but it has potential to be so much more. Imagine if it was possible for anyone to set up a distributed, fault tolerant, scalable Wordpress instance using Cloud Spanner, Cloud Functions and Cloud CDN any pay per uncached pageload? You could even set it up with a single click in the Cloud Console. Running your own scalable cloud service would be no more difficult than setting up a Tumblr blog.

Re:Cloud Spanner is now production-ready by boulos on Tuesday May 16, 2017 @06:38PM (#14353931)
Ha! I calculated the monthly cost as $.9/hr x 730 hrs => $657. You said $648, so that left $9. Sorry for the confusion!

But yes, we already have the per-op model for Cloud Datastore (and it's part of what makes it so attractive!) so you can understand that we're naturally inclined to do just what you're saying. Making Spanner multitenant in a secure, performant manner is real work. But it's what Datastore, Big Query and our other shared services do. I hope you can appreciate that getting "Dedicated Spanner" out the door was the right first step though.

Re:Cloud Spanner is now production-ready by Veratyr on Tuesday May 16, 2017 @07:05PM (#14354123)
Ah, I just used 720h :)

And that's fair enough, I understand dedicated/shared are very different things, it's just that for me, right now, Spanner isn't viable. Change that and you'll be a whole lot more appealing!

Re:Cloud Spanner is now production-ready by narrowingorbits on Tuesday May 16, 2017 @03:21PM (#14352440)
This makes sense. But, for many (at least bootstrapping) startups, $1800/mo for the database is not in the realm of the possible. Is not having to deal with (future) sharding headaches "worth" that much? Sure. But that doesn't mean it's possible.

So by pricing us out of the entry level, you will forgo having startups build their platforms on it and get hooked (while it's still at 1-node mysql scale).

When the sharding headache hits and they're scrambling to scale, will they switch to Spanner? Maybe. Probably. They'll want to, anyway. But there is a big market that will be missed.

Re:Cloud Spanner is now production-ready by matt2000 on Tuesday May 16, 2017 @03:52PM (#14352706)
I think I agree with you. Most databases these days get adopted by being open source, or pretty inexpensive for hobby projects. That lets engineers experiment with them, get comfortable and then recommend them for a major project. I'm not even going to get started with spanner, so I'm not sure how I would build up the confidence to use it in a major deployment.
Re:Cloud Spanner is now production-ready by time4tea on Tuesday May 16, 2017 @04:20PM (#14352955)
Really?? How much does a person cost per month? You could avoid lots of fiddling for many people just by using appropriate tools. Applies for everything, not just this.
Re:Cloud Spanner is now production-ready by tdfx on Tuesday May 16, 2017 @10:18PM (#14355089)
But that person needs to know the appropriate tool to use, given their experience. They are going to gravitate towards services they're familiar with, and by pricing things on the high end, you'll reduce the number of people who get to play with it.
Re:Cloud Spanner is now production-ready by narrowingorbits on Tuesday May 16, 2017 @04:40PM (#14353117)
As I said, that makes sense, and I agree. That does not mean it's financially possible.
Re:Cloud Spanner is now production-ready by derefr on Tuesday May 16, 2017 @07:43PM (#14354361)
I think the underlying problem here is that there's no "mock Spanner" (in the Minio-is-mock-S3 sense) for people to write code against on-and-off for a while, to decide whether they like what Spanner does for them in development workflow terms.

Two node-weeks of Spanner credit, used wisely, might be enough to tell you whether Spanner fits your operations requirements, but it won't be enough to tell you whether you want to commit to creating an application architected around the paradigm it represents.

Re:Cloud Spanner is now production-ready by degenerate on Tuesday May 16, 2017 @02:56PM (#14352260)
I think he edited his post so your $9/mo math no longer references anything he said.

But I think the issue is: making the commitment to use Spanner will take more than 2 weeks ($300 in credit) to determine if it's worth using.

Why not setup a "shared Spanner" with millions of pre-populated rows, and let free accounts have unlimited read-only access to it? That would let people fiddle with the technology before sweating bullets during the 2 week trial. (5 day trial if you run the recommended 3 node minimum)

Re:Cloud Spanner is now production-ready by jbergens on Wednesday May 17, 2017 @02:29PM (#14361332)
They could also increase the credit for testing. For example if you are a verified company and want to try it they could give you 2-3 months free or really cheap.
Re:Cloud Spanner is now production-ready by manigandham on Tuesday May 16, 2017 @05:46PM (#14353590)
It's understandable and not expensive considering the quality of the offering - however startups and even teams within large companies prefer and have gotten used to seamless easy scaling from small accounts so that they can test and like the product first before committing to it completely - especially one that is built into GCP and not portable.
Re:Cloud Spanner is now production-ready by crypto5 on Tuesday May 16, 2017 @02:13PM (#14351898)
And what are the specs for this node? CPU, RAM?
Re:Cloud Spanner is now production-ready by oaktowner on Tuesday May 16, 2017 @03:38PM (#14352582)
According to this article[0] (that someone referenced below):

A Cloud Spanner “node” isn't a physical machine, per se. It is an allocation of compute capacity and disk access throughput on a set of machines with access to the storage API.

(disclosure: I work at Google Cloud but not on Spanner)

[0] https://quizlet.com/blog/quizlet-cloud-spanner

Re:Cloud Spanner is now production-ready by puzzle on Tuesday May 16, 2017 @10:07PM (#14355038)
Cloud Bigtable is similar.
Re:Cloud Spanner is now production-ready by Veratyr on Tuesday May 16, 2017 @02:16PM (#14351933)
They don't appear to document this, save that they handle 10k QPS read or 2k QPS write and can deal with 2TiB of data. They also recommend that you have 3 of them:

https://cloud.google.com/spanner/docs/instance-configuration

Re:Cloud Spanner is now production-ready by kyrra on Tuesday May 16, 2017 @01:10PM (#14351269)
https://quizlet.com/blog/quizlet-cloud-spanner

For lower QPS, normal RDBMS solutions (like mysql) have lower latency than Spanner by a decent amount. But Spanner scales way beyond what MySQL is capable, which is where it starts to shine.

Re:Cloud Spanner is now production-ready by tylerpachal on Tuesday May 16, 2017 @01:57PM (#14351742)
That was a good article, thanks!
Re:Cloud Spanner is now production-ready by jlg23 on Tuesday May 16, 2017 @02:17PM (#14351946)
Because your locally hosted DB will run circles around it. And for 99.99999% of the shops out there a single, local DB instance is more than enough.
Re:Cloud Spanner is now production-ready by mentat on Tuesday May 16, 2017 @02:45PM (#14352170)
Until the backup doesn't work or it does but it takes a day to recovery it or whatever. DBs aren't free, ever.
[dead]
Re:Cloud Spanner is now production-ready by Jgrubb on Tuesday May 16, 2017 @09:32PM (#14354895)
Also possible that it wasn't meant to be taken literally.
Re:Cloud Spanner is now production-ready by ReidZB on Tuesday May 16, 2017 @01:07PM (#14351234)
In addition to the other comments, vendor lock-in. My shop is strictly AWS, so...
Re:Cloud Spanner is now production-ready by chaostheory on Tuesday May 16, 2017 @01:15PM (#14351318)
I kind of agree...

> including ANSI 2011 SQL support

which kind of eases this concern a bit. If Azure Cosmos' marketing emphasized this more, this wouldnt be as big of a concern for it either.

Re:Cloud Spanner is now production-ready by thesandlord on Tuesday May 16, 2017 @01:50PM (#14351669)
The SQL support is currently for queries only. DML is not supported (yet), so keep that in mind.

(I work for Google Cloud)

Side note: Cosmos is really confusing to me, it has 5 consistency models, supports 4 different database models (I don't think relational is an option?), multiple different APIs, etc.

For example, Azure advertises the p99 latency, but don't specify the consistent model. I'm thinking it's for the eventual consistency, so in that case what's the latency for the strong consistency model? Definitely need to learn more.

Re:Cloud Spanner is now production-ready by jdhawk on Tuesday May 16, 2017 @01:05PM (#14351222)
No DML Statements prevent quick migrations.
Re:Cloud Spanner is now production-ready by zzzcpan on Wednesday May 17, 2017 @08:13AM (#14357626)
The simple reason it isn't the RDBMS holy grail is that most project that rely on RDBMSs don't really care about CAP or strong consistency and once at scale the whole RDBMS thing just doesn't matter anymore, but things like latency usually do matter and it does sacrifice latency, despite bold claims of googlers.

I think Cockroach suits the RDBMS world much better. It understands that there is little reason to make it scale beyond a small amount of nodes in couple of nearby datacenters, but it does offer a lot compared to pre-CAP databases, meaning that it will be cheaper to operate and with no cloud lock-in risk.

Re:Cloud Spanner is now production-ready by areefer on Wednesday May 17, 2017 @09:08AM (#14358020)
There are a set of RDBMS use cases that do care about CAP and consistency - specifically applications that need Active-Active or Multi-Master capabilities. These use cases need scale across multiple data centers and low latency transactions.

But CockroachDB suffers from worse latency limitations than Spanner since it relies on NTP/hybrid clock to serialize transactions.

Disclosure - I work for NuoDB.

Re:Cloud Spanner is now production-ready by tyingq on Tuesday May 16, 2017 @04:29PM (#14353025)
No SQL insert/update plus hard lock-in via egress charges and now incompatible app changes. Work hard to port your app to get there, work/pay hard to get it out.

Technically stunning for sure though.

Re:Cloud Spanner is now production-ready by pbhowmic on Tuesday May 16, 2017 @10:53PM (#14355260)
It is very good as far disk-based databases go but relational databases have moved to the point of main memory only and NVRAM databases. And that's where the current action really is.
Re:Cloud Spanner is now production-ready by dnackoul on Tuesday May 16, 2017 @07:05PM (#14354127)
Also, how does Spanner get around the CAP theorem and similar distributed systems results? It seems like the only tradeoff is monetary at the moment.
Re:Cloud Spanner is now production-ready by brown9-2 on Tuesday May 16, 2017 @07:46PM (#14354374)
Re:Cloud Spanner is now production-ready by manigandham on Tuesday May 16, 2017 @10:23PM (#14355118)
The only important line, which means it's a CP system:

> Even then outages will occur, in which case Spanner chooses consistency over availability.

Re:Cloud Spanner is now production-ready by dnackoul on Tuesday May 16, 2017 @08:13PM (#14354517)
Thanks!
Re:Cloud Spanner is now production-ready by manigandham on Tuesday May 16, 2017 @10:21PM (#14355107)
It doesnt - they're saying they have such good networking that P is very rare so it's one of the few times you treat it like it's CA.

In case P does happen (a network break), it's a CP system.

Re:Cloud Spanner is now production-ready by kyledrake on Tuesday May 16, 2017 @02:09PM (#14351859)
It's too expensive.
Re:Cloud Spanner is now production-ready by foobarbazetc on Tuesday May 16, 2017 @02:06PM (#14351836)
Massive lock in.
Re:Cloud Spanner is now production-ready by Thaxll on Tuesday May 16, 2017 @01:02PM (#14351196)
It is but it's quiet expensive.
Re:Cloud Spanner is now production-ready by forgot-my-pw on Tuesday May 16, 2017 @01:28PM (#14351456)
It's the first in the market and pretty groundbreaking tech. Alternatively, you can use CockroachDB, TiDB, or other open source solutions, but you will need to manage your own hardware/cloud.
Re:Cloud Spanner is now production-ready by kyledrake on Tuesday May 16, 2017 @02:17PM (#14351947)
Right. That's the choice cash-strapped startups will make instead of using this. And once they're experts at that particular problem domain, they will figure out how to make their first choice redundant/HA enough that they probably won't need what spanner provides.

I used Linux instead of Windows NT as a kid because it was cheaper and ran well on cheaper hardware. Then I became an expert at it. Then it became the OS I used for servers.

I've wanted to use spanner for years, but at this point, the longest my production SQL database will ever be down is for less than 2-3 minutes, and it's not worth more than the cost of a rack full of servers and a massive migration effort to theoretically shave off a couple minutes of potential downtime for me. If I'm going to go that route, I would probably just opt for something like DRBD. The way my current infrastructure is set up, it wouldn't add any costs for me to do this.

Re:Cloud Spanner is now production-ready by Thaxll on Tuesday May 16, 2017 @02:39PM (#14352124)
Those are not even close to Cloud Spanner, what make Spanner great is all the Google backend reliability that you don't have to manage.
Re:Cloud Spanner is now production-ready by aoeusnth1 on Tuesday May 16, 2017 @05:58PM (#14353690)
You can run CockroachDB on GCE instances.
Re:Cloud Spanner is now production-ready by nikanj on Tuesday May 16, 2017 @04:40PM (#14353127)
Because it comes equipped with Google's support offering, i.e. "godspeed, and maybe we'll get back to you if your blog post makes it to the front page of a news aggregator". Companies pay real money to have real support from companies such as Microsoft. Look at https://blogs.msdn.microsoft.com/ntdebugging/ for example, they sink ungodly amounts of resources helping customers debug obscure corner cases.
Re:Cloud Spanner is now production-ready by manigandham on Tuesday May 16, 2017 @05:49PM (#14353608)
Are you an actual paying GCP customer with a support contract?

There will always be some cases of bad customer service but our support interactions have always been quick, relevant and helpful.

Re:Cloud Spanner is now production-ready by nikanj on Wednesday May 17, 2017 @10:44AM (#14358885)
As there's a story like [1] on the front page of HN weekly if not daily, I've done my best to avoid building anything on their platform. To quote [2], "They have no phone number to contact, no way to dispute this other than email — which they have ignored us for over a month now without replying to our continued requests."

https://news.ycombinator.com/item?id=14356409 https://medium.com/@contact_16315/firebase-costs-increased-b...